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SECTION  ONE 


INTRODUCTION 


THE  PROBLEM 


The  costs  associated  with  the  operation  of  major  Navy  systems  make 
increased  dependence  on  simulators  and  other  types  of  training  devices  for 
achieving  and  maintaining  operational  readiness  an  economic  imperative.  This 
is  particularly  true  in  the  area  of  flight  training,  although  it  can  be  said 
of  other  major  systems  as  well.  For  example,  in  a  document  issued  by  the 
Chief  of  Naval  Operations,  cited  by  Stoffer,  Blaiwes,  and  Brictson  (1980),  a 
plan  was  formulated  for  achieving  a  25  percent  reduction  in  Navy  flight  hours, 
and  consequent  fuel  savings,  by  the  end  of  1981.  The  flight  simulator  is 
seen,  by  some  higher  command  echelons  in  the  Navy,  as  the  potential  vehicle 
for  flight  substitution  in  achieving  and  maintaining  necessary  operational 
skills.  Traditionally,  however,  local  training  commands  have  viewed  the 
simulator  as  a  means  for  supplementing  rather  than  replacing  aircraft 
training.  According  to  Stoffer,  et  al.,  (1980)  (citing  Navy  Audit  Review  24) 
this  type  of  discrepancy  in  the  perceived  proper  role  of  flight  simulators  is 
a  major  reason  for  documented  underutilisation  of  aviation  training  devices 
and  the  consequent  failure  to  demonstrate  cost  savings  through  flight  hour 
reductions . 

If  training  devices  are  to  assume  increasing  roles  as  substitutes  for  the 
use  of  operational  equipment  in  meeting  training  objectives,  it  is  clear  that 
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they  must  be  accepted  by  Navy  personnel  in  that  role.  This  is  a  particularly 
complex  issue  when  the  training  device  is  used  for  refresher  training  or 
advanced  skill  development  as  opposed  to  initial  skills  training.  Mackie, 
Kelley,  Moe,  and  Mecherikoff  (1972),  in  an  earlier  study  of  factors 
influencing  the  acceptance  and  use  of  Navy  training  devices,  observed  that  the 
level  of  user  experience  in  the  area  of  operations  represented  by  a  trainer 
was  often  a  major  factor  influencing  acceptance.  In  general,  if  the  trainer 
was  seen  as  having  simulation  deficiencies,  experienced  personnel  were  more 
critical  of  it  than  were  less  experienced  personnel.  However,  inexperienced 
personnel  depreciated  the  value  of  a  trainer  because,  in  their  view,  its  use 
conflicted  with  the  opportunity  to  "do  the  real  thing." 

It  is  evident  from  the  work  of  Mackie,  et  al.,  (1972)  and  from  a  more 
recent  investigation  by  Caro,  Shelnutt,  and  Spears  of  the  utilization  of  Air 
Force  aircrew  training  devices  (1980)  that  training  device  acceptance  is  a 
function  of  a  multiplicity  of  factors,  some  having  to  do  with  the  fidelity  of 
simulation  of  the  equipment  and  operating  environment,  some  having  to  do  with 
design  factors  that  facilitate  or  impede  the  instructional  process,  some 
having  to  do  with  the  quality  of  instructor  personnel  and  their  attitudes 
toward  the  usefulness  of  training  devices,  some  having  to  do  with  the 
perceived  role  of  the  training  device  within  the  framework  of  a  broader 
training  program,  some  having  to  do  with  trainer  reliability,  and  some  having 
to  do  with  organizational  attitudes  and  procedures,  both  local  and  at  higher 
echelons  in  the  command  structure.  Deficiencies  in  any  of  these  areas  can 
lead  to  under-utilization,  inappropriate  utilization,  or  outright  rejection  of 
training  devices  (Mackie,  1975). 
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Caro,  et  al.,  (1980)  cited  several  studies  which  indicate  that  aircrew 
training  devices  (ATDs)  are  used  more  effectively  when  there  is  a  dominent 
positive  attitude  toward  their  use  in  training,  whereas  negative  attitudes 
prevail  in  units  where  ATDs  are  less  effectively  used.  The  question  of 
whether  the  attitude  precedes  the  pattern  of  use  or  follows  it  may  be  moot, 
but  the  association  is  sufficiently  strong  that  an  understanding  of  the 
variables  affecting  attitude  formation  in  the  context  of  trainer  use  would 
seem  essential  to  the  Navy's  interests.  Stoffer,  et  al.,  (1980)  have 
emphasized  that  increased  acceptance  of  ATDs  is  especially  critical  at  this 
time  because: 


1.  Significant  advances  in  effective  training  technology 
remain  to  be  incorporated  into  the  Fleet,  some  of  which 
may  involve  problems  of  innovation  acceptance. 

2.  A  variety  of  low  cost,  low  fidelity  approaches  to  training 
system  design  are  under  development  which  may  encounter 
user  resistance  because  of  the  traditional  emphasis  on 
high  physical  fidelity. 

3.  There  is  increasing  need  to  introduce  training 

improvements  and  have  them  accepted  as  early  as  possible, 
to  avoid  the  steadily  increasing  delay  time  in  making 
needed  training  capabilities  available  in  conjunction  with 
the  delivery  of  major  weapon  systems. 

4.  Training  technology  will  be  increasingly  integrated  into 
Fleet  operational  systems.  With  the  advent  of  compact 
portable  equipment  that  can  be  used  aboard  ship,  as  well 
as  training  devices  imbedded  in  actual  operational 
equipment,  special  problems  of  acceptance  and  use  may  be 
antici pated. 

5.  A  substantial  reduction  in  the  number  of  hours  available 
for  using  actual  operational  equipment  for  training 
purposes  is  likely,  if  not  certain. 

6.  Operational  tasks  continue  to  increase  in  complexity,  and 
the  consequences  of  human  error  or  deficient  performance 
are  increasingly  costly. 
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TOWARD  A  SOLUTION 


The  purpose  of  this  study  was  to  develop  increased  understanding  of  the 
individual,  group,  organizational,  and  hardware  factors  that  determine  the 
acceptance  and  optimal  utilization  of  innovations  in  training  technology.  It 
is  evident  that  the  Navy  will  undergo  a  period  of  increased  dependence  in  the 
1980's  on  various  types  of  innovative  training  devices.  Many  of  these 
devices,  espeically  those  involving  new  technology  and,  in  some  cases,  low 
physical  fidelity,  will  likely  pose  problems  of  innovation  acceptance  on  the 
part  of  both  individuals  and  Navy  organizations. 

A  product  of  the  study  was  a  predictive  model  of  the  process  of  acceptance 
(or  rejection)  of  training  devices  which  reflects  some  of  the  theoretical 
constructs  of  organizational  change  and  the  psychology  of  innovation 
acceptance.  An  objective  of  the  project  was  the  operationalization  of  those 
constructs  in  terms  of  variables  found  to  operate  at  all  levels  in  the  Navy 
that  impact  the  acceptance  and  use  of  training  devices  in  their  intended 
roles.  We  viewed  the  study  as  a  problem  in  organizational  psychology  because, 
although  actual  training  device  usage  is  undoubtedly  influenced  by  many  design 
and  pedagogical  factors,  organizational  change  is  necessary  at  several  levels 
in  the  Navy  if  the  broadened  role  of  training  devices  in  achieving  operational 
readiness  is  to  be  realized. 

Mackie  (1S75)  has  pointed  out  that  no  trainer  is  totally  "accepted"  or 
"unaccepted."  However,  there  is  little  doubt  that  the  acceptance  of  training 
devices  can  be  increased  with  the  consequence  of  both  improved  training  and 
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greater  cost-effectiveness.  The  problem  is  how  to  accomplish  this  objective 
in  a  systemtic  manner  that  is  also  cost-effective.  To  date,  there  has  been  no 
systematic  approach  or  plan  for  insuring  user  acceptance  (Stoffer,  et  al., 
1980).  The  result  is  that  neither  the  developers  nor  the  users  of  trainers 
know  now  to  handle  acceptance  problems  when  they  occur.  Further,  there  is  no 
basis,  at  this  time,  for  predicting  the  impact  of  particular  training  device 

features  that  will  serve  to  enhance  or  degrade  user  acceptance  other  than  the 

general  dictum  that  acceptance  can  be  "bought"  through  a  high  degree  of 
physical  fidelity,  an  approach  that  may  not  enhance  training  but  will 
certainly  prove  costly. 

In  an  effort  to  move  toward  a  solution  to  these  problems,  research  was 
conducted  to  develop  a  model  of  factors  influencing  user  acceptance  based  on 
(1)  results  of  studies  previously  conducted  specifically  on  the  trainer 
acceptance  issue;  (2)  data  from  comprehensive  interviews  with  various 

"players"  in  the  innovation  and  acceptance  process;  and  (3)  previous 

theoretical  modeling  of  organizational  change  and  innovation  acceptance.  The 
next  section  describes  useful  concepts  from  a  review  of  the  literature  on 
innovation  acceptance,  technology  transfer,  and  organizational  development. 
Section  3  provides  an  overview  of  the  Navy's  training  device  acquisition 
processes.  Section  4  presents  the  model  which  was  developed,  and  Section  5 
provides  an  interpretation  of  interview  results  in  the  context  of  the  model. 
The  text  is  supported  by  detailed  material  presented  in  the  Appendices. 
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SECTION  TWO 


REVIEW  OF  THE  LITERATURE 


INNOVATION  ACCEPTANCE 


In  general,  the  acceptance  of  new  training  devices  and  new  training 
technology  was  viewed  as  a  special  case  of  the  general  problem  of  innovation 
acceptance.  Indeed,  the  problems  associated  with  low  cost,  "low  fidelity"^ 
trainers  would  appear  to  pose  a  particular  challenge  to  the  application  of 
innovation  acceptance  theory. 

The  innovation-decision  process  has  been  described  in  the  literature  as 
the  mental  process  through  which  an  individual  passes,  starting  with  his  first 
knowledge  of  an  innovation,  to  an  eventual  decision  to  adopt  or  reject  (Rogers 
and  Shoemaker,  1971).  The  stages  in  this  process  have  been  described  by  many 
researchers  and  include  (1)  awareness  (first  knowledge  of  the  innovation),  (2) 
interest  (gaining  further  knowledge  about  it),  (3)  evaluation  (gaining  a 
favorable  or  unfavorable  attitude  toward  the  innovation,  (4)  small  scale 
trial,  and  (5)  decision  to  adopt  or  reject.  Rogers  and  Shoemaker  call  the 
final  stage  "confirmation,"  feeling  that  this  allows  for  post-decision 
communication  behavior  which  usually  reinforces  the  original  decision  but  may 
lead  to  its  reversal . 


1  Fidelity  in  this  context  refers  to  the  degree  of  physical  and  functional 
resemblance  between  the  trainer  and  the  operational  system.  High  fidelity  may 
be  neither  necessary  nor  sufficient  for  effective  training  of  particular  tasks. 


* 
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FURTHER  INFORMATION  SEEKING 

REVISION  OF  EVALUATION 


j 


ON 


Figure  1  diagrams  the  dynamics  of  the  innovation-decision  process  as  we 
hypothesize  it  might  occur  during  the  introduction  of  new  training 
technology.  We  will  elaborate  on  this  diagram  in  some  detail. 

Initial  Awareness 


Initial  awareness  of  a  new  device  or  training  technique  is  stimulated  in 
Navy  personnel  through  a  variety  of  channels,  both  formal  and  informal.  It  is 
important  to  recognize  that  if  the  training  system  is  truly  innovative,  i.e., 
it  is  not  simply  an  improved  version  of  an  older  system,  the  content  of  this 
initial  communication  can  be  very  important  to  acceptance.  There  is  an 
important  distinction  in  this  regard  between  personnel  who  may  be  in  their 
initial  stages  of  training  and  who  have  no  prior  experience  with  training 
devices,  and  those  who  are  undergoing  refresher  training  or  advanced  tactical 
training.  The  latter  personnel  may  have  a  far  different  perspective,  as 
previously  noted,  about  the  role  that  training  devices  should  play  in  the 
total  training  program  than  the  former. 

Whatever  the  nature  of  the  initial  communication,  it  is  important  that  it 
be  accurate  and  reasonably  comprehensive.  In  military  organizations, 
remarkable  word  of  mouth  inaccuracies  can  pervade  initial  awareness. 
Obviously,  any  such  distortions  can  adversely  impact  the  acceptance  process. 

In  the  diagram,  we  have  shown  a  "trainer  advocate"  (TA)  making  an  input  to 
initial  awareness.  The  TA  is  modeled  after  the  "change  advocate,"  a 
professional  who,  in  various  ways,  influences  innovation  decisions  by  means  of 
direct  interactions  with  user  personnel.  Though  the  change  advocate  has  had  a 
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long  and  historically  important  role  in  promoting  the  adoption  of  innovations 
in  other  contexts,  he  is  rarely  if  ever  evident  during  the  process  of 
introducing  new  systems  to  military  organizations.  (For  this  reason,  dashed 
lines  are  used  to  show  the  areas  of  his  impact  in  the  figure).  Nevertheless, 
earlier  studies  (Mecherikoff  and  Mackie,  1970;  Mackie,  1  975;  Caro,  et  al . , 

1980)  have  emphasized  the  importance  of  the  advocacy  role.  It  is  notable  that 
both  Navy  and  Air  Force  studies,  conducted  independently,  identified  of 
informal  advocacy  by  a  few  highly  motivated  individuals,  with  very  desirable 
trainer  acceptance  and  use  outcomes.  Usually  this  has  occurred  without 

official  direction  by  local  commands. 

Immediate  Perception  of  Need 

Whatever  the  source  of  the  initial  communication,  the  user's  inroediate 
perception  of  the  need  for  a  training  innovation  will  be  a  function  of  the 
problems  he  has  experienced  in  the  operational  arena.  It  is,  unfortunately,  a 
simplification  to  assume  that  all  potential  users  will  have  the  same 

appreciation  of  the  operational  problem.  There  is  also  a  risk  that  some  users 

will  view  the  development  agency  as  not  fully  perceptive  of  the  operational 
problem  and  that  this  will  be  reflected  in  trainer  design  deficiencies.  It  is 
also  likely,  of  course,  that  different  categories  of  users  (e.g., 

undergraduate  pilots  versus  experienced  ones)  will  have  quite  different 

perceptions  of  the  need. 

One  of  the  complicating  factors  in  the  introduction  of  many  innovations  is 
that  the  intended  users  may  not  be  particularly  aware  of  the  need  that 

stimulated  its  development.  This  problem  is  frequently  encountered  as  a 
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consequence  of  high  turnover  rates  among  military  personnel.  During  the  time 
required  for  trainer  development,  the  perception  of  need  may  change 
considerably,  either  because  of  individual  differences  in  perspective  or 
because  of  actual  changes  in  operational  requirements.  This  is  perhaps  less 
of  a  problem  in  the  case  of  system  simulators  because  of  their  clear 
relationship  to  their  operational  counterparts.  Nevertheless,  there  are  two 
major  roles  for  the  trainer  advocate  at  this  stage:  (1)  He  can  be  the  source 
of  valid  information  concerning  current  and  future  operational  problems 
associated  with  the  new  trainer  development  and  (2)  He  can  be  the  source  of 
valid  information  concerning  how  the  innovation  is  expected  to  aid  in  solving 
those  problems. 

Level  Of  Interest 


Initial  level  of  interest  is  clearly  a  function  of  the  user's  perception 
of  need  for  improvement  and  a  general  awareness  of  the  purpose  of  the  training 
innovation.  If  he  personally  identifies  with  the  operational  problems  the 
trainer  is  designed  to  address,  or  if  the  trainer  advocate  has  done  his  job  in 
making  him  aware  of  those  problems,  his  initial  level  of  interest  should  be 
high  enough  that  he  will  at  least  be  receptive  to  further  information  about 
the  innovation,  even  if,  at  this  stage,  he  does  not  actively  seek  it.  On  the 
other  hand,  if  his  perception  of  the  need  for  improvement  is  low,  no  further 
information  seeking  or  reception  may  occur. 
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Information  Acquisition 


Assuming  some  degree  of  information  seeking  or,  at  least  receptivity, 
information  received  through  personal  communication  channels  is  likely  to  be 
more  persuasive  at  this  early  stage  than  that  received  through  other  media 
(Rogers  and  Shoemaker,  1971).  The  important  point  to  consider  is  that,  in  the 
absence  of  authoritative  messages,  the  user's  general  perception  of  the 
innovative  system  can  be  a  function  of  informal  channels  that  may  contain  a 
good  deal  of  "noise."  Indeed,  there  is  no  assurance  that  authoritative 
information  will  be  provided  at  this  stage  in  the  absence  of  controlled 
presentations  by  the  development  agency  or  a  trainer  advocate.  Further,  it 
should  be  recognized  that,  in  the  absence  of  authoritative  sources,  the 
information  void  will  be  filled  through  informal  channels.  The  likelihood 
that  certain  misconceptions  will  develop  through  these  channels  seems 
particularly  high  when  the  innovative  system  is  viewed  as  intrusive  on  other, 
more  established  training  procedures. 


^The  term  "user"  is  employed  here  in  a  broad  sense.  It  refers  not  only  to 
students  who  will  utilize  the  trainer  but  to  instructor  personnel  who  may  be 
assigned  to  operate  it,  as  well  as  to  administrative  personnel  and  those 
further  up  in  the  training  command  and/or  Fleet  hierarchy  with 
responsibilities  for  training  and  readiness. 
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Perceived  Features  and  Perceived  Need 


Whatever  its  sources,  the  initial  information  is  responsible  for  the 
user's  early  perception  of  the  potential  usefulness  of  the  new  development  to 
him.  There  follows  a  general  more  or  less  favorable  impression  depending  on 
how  congruent  this  perception  is  with  the  user's  view  of  the  operational 
problem  and  felt  need  for  improvement.  This  initial  reaction  may  be  based  on 
more  or  less  detail  concerning  various  features  of  the  innovation  and,  indeed, 
congruence  may  increase  or  decrease  as  further  information  is  acquired.  As 
indicated  in  the  Figure  1  the  process  is  viewed  as  interactive;  subsequently 
received  information  can  result  in  either  increasing  or  decreasing  the  match 
between  perceived  need  and  various  perceived  features  of  the  innovation. 

Subjective  Evaluation 


As  Stoffer,  et.al.,  (1980,  op.cit.)  have  noted,  the  criteria  of  acceptance 
at  the  user  level  are  based  on  a  tradition  which  requires  the  user  to  make 
primarily  subjective  determinations  of  training  value.  In  their  view,  as  well 
as  ours,  subjective  opinion  will  continue  to  be  a  primary  determiner  of 
innovation  acceptance.  Although  they  note  that  subjective  acceptance  of 
training  equipment  does  not  guarantee  optimal  training  value,  it  is 
nonetheless  true  that  the  absence  of  a  positive  subjective  evaluation  will 
limit  effective  use  of  the  trainer  regardless  of  its  theoretical  advantages. 
Because  of  the  central  role  of  subjective  evaluation  in  the  acceptance 
process,  several  postulated  aspects  of  the  process,  as  shown  in  Figure  1,  will 
be  briefly  described. 
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Relative  Advantage  is  defined  as  the  degree  to  which  an  innovation  is 
perceived  as  better  than  the  idea  or  device  it  is  intended  to  supercede.  The 
relative  advantage  of  an  innovation,  as  viewed  by  members  of  the  user  group, 
is  positively  related  to  its  rate  of  adoption  (Rogers  and  Shoemaker,  1 971 ) 

It  probably  follows  that  any  innovation  that  is  not  seen  as  having 
relative  advantage  has  little  prospect  of  adoption  in  the  Navy.  However,  even 
when  there  is  evidence  of  relative  advantage,  adoption  is  not  insured  because 
many  other  practical  considerations  enter  into  the  acceptance-rejection 
decision.  The  identification  of  these  considerations,  some  of  which  are 
discussed  later,  should  be  one  of  the  main  concerns  during  the  process  of 
introducing  innovations. 

Compatibility  is  defined  by  theorists  as  the  degree  to  which  an  innovation 
is  perceived  to  be  consistent  with  the  existing  values,  past  experience,  and 
needs  of  the  users.  In  addition,  Navy  personnel  are  concerned  with 
compatibility  in  the  sense  that  an  innovation  must  be  operationally  compatible 
with  other  systems  with  which  it  must  work.  (This  point  is  germane  to  the 
concept  of  embedded  trainers  or  on-board  trainers  of  any  type.) 
Innovation-acceptance  theory  suggests  that  the  compatibility  of  an  innovation, 
as  perceived  by  its  users,  is  positively  related  to  its  rate  of  adoption 
(Rogers  and  Shoemaker,  1S71). 

Complexity  is  defined  as  the  degree  to  which  an  innovation  is  perceived  as 
relatively  difficult  to  understand  and  use.  The  complexity  of  an  innovation, 
as  perceived  by  users  of  the  system,  is  generally  regarded  as  negatively 
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related  to  its  rate  of  adoption  (Rogers  and  Shoemaker,  1971).  However,  there 
are  some  subtleties  involved.  An  innovative  system  may  experience  rejection 
if  it  is  seen  as  oversimpl  i fyi  ng  a  problem  that  is  known  to  be  inherently 
complex  (Mackie,  1980). 

Observabili ty  is  the  degree  to  which  the  resul ts  of  the  use  of  an 
innovation  are  visible  to  others.  The  observability  of  an  innovation,  as 
perceived  by  the  users  of  the  system,  is  considered  to  be  positively  related 
to  its  rate  of  adoption  (Rogers  and  Shoemaker,  1971).  A  demonstration  of 
positive  results  is  likely  to  be  particularly  important  in  stimulating 
acceptance  of  innovative  devices  about  which  there  is  initial  skepticism. 

Tri al ability  is  the  degree  to  which  an  innovation  may  be  experimented  with 
by  user  personnel  on  a  limited  basis.  The  trialability  of  an  innovation,  as 
perceived  by  users  of  the  system,  is  considered  to  be  positively  related  to 
its  rate  of  adoption.  It  should  be  noted  that  actual  trial  may  occur,  as  in  a 
demonstration,  or  it  may  be  vicarious;  that  is,  the  user  may  simply  visualize 
how  the  innovation  might  be  used  in  his  own  operating  environment,  by  himself 
or  others. 

It  is  evident  that  the  first  three  aspects  of  subjective  evaluation, 
outlined  above,  pertain  primarily  to  the  design  of  the  innovation  and  the 
latter  two  relate  to  how  it  is  introduced  and/or  demonstrated.  The  importance 
of  the  system  advocate  in  providing  information  and  demonstrations  cannot  be 
overestimated  for  new  systems  where  user  personnel  will  have  an  option  as  to 
whether  or  not  to  adopt.  There  is  an  important  distinction  in  this  respect 
between  new  systems  that  must  be  used  (sensor  systems,  weapon  systems. 
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navigation  systems,  etc.),  because  they  are  essential  to  mission 
accomplishment,  and  systems  such  as  trainers  whose  use  may,  rightly  or 
wrongly,  be  regarded  as  optional  by  the  intended  users. 

Thus  far  we  have  discussed  the  inputs  to  subjective  evaluations  that 
reflect  users'  perceptions  of  various  features  of  the  innovation  itself, 
augmented  or  not  by  actual  observations  and  trial.  There  are,  however,  a 
number  of  other  very  important  external  influences  on  subjective  evaluation 
which  are  quite  apart  from  the  innovative  device  itself. 

Experience  With  Similar  Developments 

The  intended  user  may  have  had  experience  with  prior  developments  that  he 
feels,  correctly  or  not,  are  similar  to  the  proposed  innovation.  If  so,  his 
subjective  evaluation  of  the  new  training  development  may  be  biased, 
positively  or  negatively,  depending  on  whether  the  prior  experience  was 
favorable  or  not.  If  there  have  been  prior  negative  experiences,  an  important 
requirement  of  the  introduction  process  is  to  counter  the  feeling  that  the  new 
development  will  not  be  any  better  than  the  last.  An  innovation  advocate  can 
play  a  critical  role  in  this  regard,  and  he  should  be  well  aware  of  the 
deficiencies  of  earlier  systems  that  are  viewed  as  belonging  to  the  same 
general  category  as  the  new  one.  It  seems  likely  that  this  can  be  a 
particularly  important  function  in  promoting  devices  such  as  flight  trainers 
since  ATDs  in  general  are  likely  to  have  gained  a  reputation  among  operational 
personnel  concerning  their  capabilities  and  limitations  in  meeting  various 
perceived  training  needs.  The  studies  of  both  Mackie,  et  al .  (1972)  and  Caro, 
et  al ,  (1980)  clearly  indicate  that  either  positive  or  negative 
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predispositions  may  be  brought  to  a  new  training  system  depending  upon  a 
variety  of  experiential  factors  associated  with  earlier  flight  trainers. 

A  related  consideration  has  to  do  with  the  technical  reputation  of  the 
developer  of  the  innovative  system.  (It  should  be  noted  that  the  developer, 
from  the  user's  perspective,  may  be  the  project  office,  a  Navy  laboratory  if 
one  is  involved,  the  Naval  Training  Equipment  Center  (NAVTRAEQUIPCEN) ,  or  the 
actual  manufacturer  of  the  equipment.)  If  the  developer  is  viewed  as  having  a 
history  of  product  development  that  is  not  well  matched  to  operational  needs, 
a  negative  predisposition  is  likely.  The  opposite  effect  can  also  occur,  if, 
of  course,  the  reputation  of  the  developer  is  generally  favorable. 

Competing  Alternatives 

It  is  rare  that  innovative  developments  are  free  from  competition  from 
alternative  ways  of  doing  things.  Except  in  the  case  of  a  totally  new  system 
development,  the  "old  way"  of  doing  things  is  almost  always  an  alternative 
that  enters  into  the  subjective  assessment  of  relative  advantage.  In  the  case 
of  ATDs,  there  is  a  special  source  of  competition,  namely,  flight  experience 
in  the  aircraft  itself  which,  on  the  basis  of  anecdotal  reports,  can  be 
expected  to  be  a  strong  biasing  factor  in  the  subjective  evaluation  process. 
Stoffer,  et  al.,  (1S80),  in  their  analysis  of  user  acceptance  of  R&D  in  Navy 
training,  listed  several  sources  of  competing  alternatives  associated  not  only 
with  individual  users,  but  with  groups  and  organizations  that  may  affect  the 
subjective  evaluation: 
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1.  Individual  knowledge  of  an  alternative  training  method 

2.  Individual  attitudinal  commitment  to  the  alternative 

3.  Individual  behavioral  commitment  to  the  alternative 

4.  Group  behavioral  commitment  to  the  procedures  called  for  by  the 
al ternative 

5.  Organization/institutional  policy  commitment  to  the  alternative 

It  is  important  to  recognize  that  the  subjective  evaluation  may  depend  on 
perceived  relative  advantage  of  an  alternative  training  approach  or  system 
whether  or  not  that  advantage  is  real  in  terms  of  its  impact  on  operational 
readiness.  The  acceptance  advantage  usually  enjoyed  by  training  systems  that 
have  a  high  degree  of  physical  resemblance  to  operational  equipment  provides 
an  example.  If  a  low  cost,  low  fidelity  simulator  is  capable  of  achieving 
specified  training  objectives  as  well  or  better  than  more  elaborate  and  more 
costly  systems,  such  systems  are  nevertheless  likely  to  be  viewed  with  an 
initial  outset  with  a  negative  bias  by  users  who  have  been  brought  up  in  the 
tradition  of  high  fidelity  simulation.  The  burden  in  such  cases  will  fall 
much  more  heavily  on  observation  and  trial  to  demonstrate  the  relative 
advantage  of  low  fidelity  systems  in  regard  to  particular  training  objectives. 

User  Participation  in  Design 

A  strong,  though  sometimes  temporary,  negative  bias  toward  innovative 
systems  is  reflected  by  the  "not  invented  here"  syndrome.  This  reaction  is 
particularly  likely  if  the  innovative  system  is  seen  as  an  intrusion  into  the 
user’s  own  area  of  technical  expertise,  which  in  the  case  of  experienced 
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operational  personnel,  is  very  likely.  If  neither  the  user  nor  other 
personnel  with  similar  qualifications  were  consulted  with  respect  to  design, 
an  initial  negative  attitude  toward  the  innovation  is  frequently  the  result. 
As  noted  in  other  studies  of  innovation  acceptance  in  the  Navy,  highly 
qualified  users  should  be  involved  in  the  design  process  if  at  all  possible 
(Mecherikoff  and  Mackie,  1970).  If  they  are  not,  the  change  advocate  may  have 
to  make  an  effort  to  secure  the  endorsements  of  a  recognized  group  of 
qualified  peers.  Caro,  et  al . ,  (1980),  cite  examples  of  user  confidence  in 
new  ATDs  in  cases  where  personnel  from  their  units  had  been  involved  in  the 
design  process.  Further,  they  cite  a  long-term  positive  subjective 
evaluation,  over  a  period  of  several  years,  in  the  case  of  a  device  whose 
design  was  known  to  have  been  based,  in  part,  on  inputs  from  some  of  the 
unit's  best  pilots.  In  other  cases,  where  ATDs  were  built  and  "dumped  over 
the  fence  at  night,"  the  current  users  felt  isolation  from  the  development 
process  and  felt  that  the  trainer  had  been  designed  by  an  anonymous  "they"  who 
"didn't  know  what  we  needed." 

Stoffer,  et  al.,  (1980),  reported  that  there  has  been  little  systematic 
application  of  this  form  of  participative  management  in  the  entire  process  of 
trainer  acquisition,  effectiveness  evaluations,  and  training  optimization 
studies  .  They  cite  as  reasons  (1)  lack  of  the  recognition  of  the  value  of 
participative  management  as  a  tool  for  gaining  acceptance  of  change;  (2)  lack 

^We  will  see  later,  however,  that  the  Navy  has  made  a  concerted  effort  to 
deal  with  this  need  through  the  creation  of  Fleet  Project  Teams.  Many 
variables  operate  to  determine  the  actual  role  and  effectiveness  of  these 
teams  (See  Section  3). 
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of  operational  definitions  as  to  how  participative  management  should  be 
implemented;  and  (3)  lack  of  formal  documentation  of  successful  applications 
of  participative  management  to  specific  military  training  organizations.  In 

spite  of  this,  they  report  there  is  considerable  empirical  support  for  the 
value  of  participative  management  as  a  tool  for  introducing  organizational 
change.  Mecherikoff  and  Mackie  ( 1 970)  support  this  view  and  cite  an 
actual  case  of  training  device  acceptance  which  was  facilitated  by  user 
participation  in  design  even  though  the  "designing"  occurred  after  the 
training  device  was  actually  built. 

Personal  Risk 


All  innovations  carry  some  degree  of  subjective  risk  to  the  potential 
user.  In  the  case  of  complex  trainers,  there  are  at  least  two  categories  of 
users  to  be  considered:  (1)  the  trainee  himself  and  (2)  the  instructor.  The 
instructor  is  likely  to  be  uncertain,  initially,  as  to  how  the  innovation  will 
affect  the  conduct  of  his  assigned  responsibilities.  Since  the  instructor's 
acceptance  of  any  innovative  training  equipment  or  technology  is  probably  a 
necessary  but  not  sufficient  condition  for  student  pilot  acceptance,  the 
manner  in  which  perceived  threats  to  his  established  expertise  are  handled 
becomes  a  critical  element  of  the  innovation  introduction  process. 

Insofar  as  students  are  concerned,  the  degree  of  subjective  risk  can 
depend  heavily  on  instructor  protocol  in  using  the  training  device.  Caro,  et 
al . ,  comment  on  the  importance  of  realism  in  training  syllabi  and  scenarios 
and  on  the  negative  effect  created  by  some  instructors  who  insert  unrealistic 
events  or  systems  failures  at  a  rate  that  make  it  impossible  to  perform  well. 


19 


Thus,  for  the  students,  use  of  the  trainer  becomes  a  source  of  embarrassment. 
Faced  with  such  situations,  even  when  the  value  of  properly  conducted  ATD 
training  was  clearly  recognized,  students  soon  developed  apathetic  and  hostile 
attitudes  toward  trainer  use  because  they  were  frustrated  by  their  failures 

(Caro,  et  al ,  1980).  On  a  related  point,  Mackie,  et  al . ,  (1972)  found  that 

users  cited  instructor  qualifications  as  a  major  factor  in  the  use  and 

acceptance  of  a  variety  of  Navy  trainers. 

Availability  of  Support 

It  is  obvious  that  any  new  Navy  system  requires  proper  documentation, 

maintenance  support,  and,  in  the  case  of  training  devices,  courseware.  The 
signficance  of  these  factors,  which  we  have  simply  labeled  "support,"  lies  in 
the  early  formation  of  attitudes  of  acceptance  or  rejection  based  on  the 

user's  prior  experience  with  the  adequancy  of  these  support  functions. 
Mackie,  et  al.,  (1972)  found  that  lack  of  adequate  support  was  a  significant 

factor  in  the  nonacceptance  of  some  Navy  training  systems.  At  the  time  of 

their  study,  however,  maintenance  problems  were  probably  more  severe  than  they 
are  today.  Caro,  et  al.,  (1980)  suggest  that  trainer  reliability  has 

increased  markedly  in  recent  years,  but  interviews  conducted  during  this  study 
(see  Section  5)  indicate  that  it  is  still  a  major  problem  with  some  trainers, 
and  markedly  affects  user  acceptance. 

Support  in  the  form  of  well  designed  courseware,  however,  is  another 
matter.  We  have  already  commented  on  the  use  of  scenarios  by  some  instructors 
that  student  personnel  feel  represent  unreasonable,  non-operationally 
realistic  demands.  Unfortunately,  with  many  trainers  there  is  no  carefully 
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designed  courseware  at  all  and  much  that  is  introduced  in  the  way  of  training 
scenarios  is  at  the  whim  of  the  instructor.  Further,  as  Mackie,  et  al.  (1972) 
found,  it  is  often  left  up  to  fleet  users  to  specify  their  own  training 
scenarios.  The  latter  approach  assumes  that  the  users  know  what  type  of 
scenario  is  most  appropriate  for  meeting  particular  training  objectives,  an 
assumption  that  is  doubtful  at  best.  It  is  not  suggested  that  the  fleet  user 
is  unable  to  identify  his  operational  problem,  but  rather  that  the  translation 
of  that  problem  into  appropriate  training  scenarios  is  not  an  easy  process  for 
personnel  who  are  not  training  specialists. 

In  the  long  run,  after  the  system  is  actually  implemented,  continuing 
support  is  absolutely  essential  to  avoid  temporary  adoption  followed  by 
subsequent  disuse  and  rejection.  It  is  an  unfortunate  commentary  that  some 

innovative  Navy  systems  have  fallen  into  disuse  because  of  lack  of  adequate 
support  despite  the  fact  that  they  were  developed  to  meet  widely  recognized 
needs. 

In  addition  to  "support"  as  defined  here,  an  innovative  training  system 
may  also  require  conceptual  reinforcement  on  a  periodic  basis.  The  Navy 

system  of  personnel  rotation,  and  relatively  short  tours  of  duty,  fosters  an 
unfortunate  kind  of  "corporate  memory  loss"  concerning  the  reasons  why 

particular  innovations  were  adopted  in  the  first  place.  In  the  case  of 
complex  training  devices,  a  consequence  of  this  may  be  a  progressive 

deterioration  in  the  level  of  sophistication  with  which  the  training  device  is 
used.  Mackie,  et  al.,(  1S72)  found  that  many  trainers  designed  to  meet  very 
complex  training  requirements  were  actually  used  in  a  way  that  relegated  them 
to  a  "procedural"  trainer,  where  the  training  objectives  actually  being  served 
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could  have  been  met  with  far  less  elaborate  and  less  costly  devices.  Some  of 
the  intended,  more  sophisticated  uses  had  simply  dropped  out  of  the  program. 

Organizational  Climate 


It  is  has  long  been  held  by  various  investigators  that  some  organizations, 
more  than  others,  provide  a  receptive  climate  for  the  introduction  and 
acceptance  of  innovations.  In  the  Navy,  it  seems  very  likely  that  individual 
commanding  officers  vary  widely  in  their  receptiveness  of  innovations,  and,  in 
cases  where  the  adoption  of  a  new  system  may  be  regarded  as  discretionary , 
this  will  strongly  influence  acceptance  and  use  at  the  unit  level. 

The  term  cl  imate  has  been  defined  as  a  set  of  properties  of  the  work 
environment  that  is  assumed  to  be  a  major  force  in  influencing  the  behavior  of 
the  personnel  in  the  system.  These  properties  include  such  variables  as 
organization  size,  structure,  leadership  pattern,  interpersonal  relationships, 
systems  complexity,  goal  direction,  and  communication  patterns  (Hamner  and 
Organ,  1978,  p.  279).  Citing  Shein  (1970),  Hamner  and  Organ  report  that  a 
"healthy"  organizational  climate  is  one  that: 

1.  Takes  in  and  communicates  information  reliably  and  validly; 

2.  Has  the  internal  flexibility  and  creativity  necessary  to  make  the 
changes  which  are  demanded  by  the  information  obtained; 

3.  Includes  integration  and  commitment  to  the  goals  of  the 
organization,  from  which  comes  the  willingness  to  change; 

4.  Provides  internal  support  and  freedom  from  threat,  since  being 
threatened  undermines  good  communication,  reduces  flexibility, 
and  stimulates  self  protection  rather  than,  concern  for  the  total 
system. 
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How  the  use  and  acceptance  of  training  innovations  might  be  affected  by 
these  and  other  variables  associated  with  Navy  organizations  at  the  unit  level 
and  throughout  higher  echelons  in  the  training  command  is  by  no  means 
certain.  Although  anecdotal  reports  suggest  that  individual  commanding 
officers  vary  widely  in  their  receptivity  to  innovations,  we  know  of  no 
objective  data  to  support  this  likely  contention. 

There  are  other  variables  apparently  related  to  the  "climate  of 

receptiveness"  which  includes  such  factors  as  group  cohesiveness,  or  the 
degree  to  which  members  of  the  group  mutually  influence  one  another;  group 
norms,  which  serve  to  control  the  behavior  of  individual  group  members;  "group 
think,"  a  characteristic  of  close  knit  groups  that  has  been  described  as 

decreasing  the  openness  of  the  group  members  to  critical  examination  of 

information  which  may  sometimes  be  unsettling;  and  the  various  behaviors  of 
group  leaders  (including  informal  ones)  that  may  influence  receptivity  to  new 
procedures  and  technology. 

In  the  Figure  1,  we  have  shown  organizational  climate  as  external  to  the 
subjective  evaluation  process.  We  believe  this  is  appropriate  because  we 

perceive  organizational  climate  as  a  thresholding  influence  which  may  affect 
the  inital  response  of  an  individual  toward  adoption  or  rejection  of  an 
innovative  system  regardless  of  his  subjective  evaluation.  For  example,  if 
his  subjective  evaluation  is  positive,  but  only  mildly  so,  his  perception  of 
an  organizational  climate  that  is  highly  receptive  might  lead  to  an  initial 
response  to  adopt.  The  same  evaluation  in  a  nonreeeptive  organizational 
climate  might  lead  to  an  initial  response  to  reject. 
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Authority  Decisions 


The  remaining  element  of  Figure  1  reflects  the  fact  that  decisions  to 

adopt  or  reject  are  often  made,  particularly  in  the  military,  by  authority. 

Authority  innovation-decisions  are  those  forced  upon  an  individual  by  someone 

in  a  superordinate  position  of  power.  In  effect,  the  individual  is  ordered  to 

adopt  or  reject  the  new  development  (Rogers  and  Shoemaker,  1971).  Obviously, 
in  a  military  organization,  authority  innovation  decisions  are  commonplace. 
However,  no  matter  what  the  source  of  the  decision,  the  users  inevitably 
evaluate  the  innovation  in  terms  of  their  own  personal  operational  needs. 
This  is  not  to  say  that  they  will  not  comply  with  the  order.  The  authority 
decision  will  be  accepted  on  the  surface.  However,  compliance  is  a  very 
different  outcome  from  adoption.  In  the  case  of  compliance,  the  change  in 
behavior  is  often  temporary,  and  continued  surveillance  and  reinforcement  are 
required  to  avoid  gradual  disuse  and  rejection.  Thus,  in  the  final  box  in 
Figure  1,  we  have  distinguished  between  an  initial  response  to  adopt  on  the 
basis  of  subjective  evaluation  versus  the  temporary  adoption  that  may  occur  in 
response  to  authority. 

A  final  comment  about  this  preliminary  conceptualization  is  in  order.  It 
will  be  noted  that  the  process  ends  with  an  initial  propensity  for  adoption  or 
rejection.  Of  course,  in  the  early  stage  of  the  introduction  of  an 
innovation,  this  initial  response  may  be  followed  by  further  information 
seeking,  revision  of  the  evaluation  in  accordance  with  new  evidence,  further 
consideration  of  competing  alternatives,  etc.  Other  models  of  the  change 
process  emphasize  that  acceptance  of  an  innovation  is  eventually  reflected  in 
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organizational  or  institutional  policy,  i.e.,  that  the  change  is 
institutionalized.  While  we  have  no  fundamental  disagreement  with  this  view, 
and  in  fact  have  seen  such  institutionalization  at  times,  it  is  worth  noting 
that  some  innovations,  as  we  previously  pointed  out,  require  continuing 
reinforcement  due  to  the  corporate  memory  loss  associated  with  the  high 
turnover  rate  of  personnel  in  key  positions,  including  those  in  training. 


TECHNOLOGY  TRANSFER 


A  second  body  of  literature  having  pertinence  to  our  project  objectives 
was  that  dealing  with  the  methodology  of  technology  transfer.  The  Navy  has 
sponsored  technology  transfer  methodology  and  effectiveness  measurement 
studies  at  the  Naval  Postgraduate  School  since  1969.  More  than  30  studies 
have  been  performed  (Rohoer  and  Buckles,  1980).  Of  particular  interest  was  a 
model  of  the  technology  transfer  process  originally  presented  in  Creighton, 
Jolly  and  Denning  (1972).  A  representation  of  this  model  is  shown  in  Figure 
2.  The  following  brief  explanations  of  the  model  factors  viewed  as 
influencing  the  transfer  of  knowledge  from  supplier  to  receiver,  are  excerpted 
from  Jolly  (1980). 


1.  Method  of  Information  Documentation.  This  refers  to  the 
format,  the  language,  the  report  complexity,  and  the 
documentation  system. 

2.  The  Distribution  System.  The  distribution  system  has  many 

forms.  The  primary  distribution  system  is  person  to 
person;  however,  consideration  needs  to  be  given  to 
journals,  direct  mail,  meetings,  conferences,  and 
workshops.  These  are  all  effective  methods  and  need  to  be 
selected  and  adjusted  to  provide  an  effective  system  in 
order  to  accomplish  the  specific  objective  of  the 

organi zati  on. 

3.  The  Formal  Organization  of  the  User.  A  comfortable 

organizational  climate  is  often  a  climate  of  stability. 
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FORMAL  FACTORS 


Figure  2.  A  Model  of  Technology  Transfer. 
(Jolly,  1980.) 
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However,  .  .  the  stable  state  as  applied  to 

organization,  is  the  enerny  of  adoptive  change  ..." 
(Schone,  1967).  Stephenson,  Ganz,  and  Erickson  (1974) 

reported  the  responses  of  109  scientists  and  engineers 
from  the  Naval  Weapons  Center,  China  Lake,  California  in 
terms  of  their  perceptions  of  management  creating 
conflicting  forces  for  change.  Forty  respondents  felt 
that  their  organization  occasionally  or  often  acted  as  a 
barrier  to  the  use  of  new  ideas. 

4.  Project  Selection  Process.  Rogers  and  Jain  (1969)  have 

shown  that  "...  a  basic  reason  for  the  lack  of  research 
utilization  is  that  the  process  is  often  begun  with  the 

research  process,  rather  than  the  client's  needs  .  .  .  )." 

There  is  an  obvious  benefit  of  increasing  the  potential 
utility  of  research  through  collaboration.  In  addition, 
users  or  receivers  become  more  committed  much  earlier  to 
the  technology  transfer  effort. 

5.  Capacity  of  the  Receiver.  This  factor  refers  to  the 

ability  and  capability  of  the  potential  user  to  utilize 
new  and/or  innovative  ideas.  There  are  three  aspects  of 
capacity  to  consider,  i.e.,  skills,  education,  and  traits. 

6.  Informal  Linkers  in  the  User  Organization.  This  refers  to 

the  presence  of,  and/or  the  effects  of  individuals  in  the 
receiving  organization  who  link  or  couple  persons  in  their 
organizations  to  outside  ideas,  concepts,  and  new 

devices.  Linkers  are  persons  who  operate  in  the  same 
organization  or  allied  organizations  (with  social  overlap) 
as  those  persons  who  will  actually  use  the  new 
technology. 

7.  Credibility  as  Viewed  by  the  Receiver.  The  receiver  will 

consciously  or  unconsciously  assign  a  credibility  to  the 
technical  information  that  relates  to  potential 

innovations.  Because  individuals  have  difficulty 

distinguishing  between  the  source  or  origin  of  the  message 
and  the  channel  that  carried  that  message,  the  individual 
will  attach  a  composite  credibility  to  a  message,  derived 
from  both  perceived  source  and  perceived  channel 

reliability.  Thus,  the  character  of  the  group  or 

individual  originating  the  idea  or  performing  the  research 
and  development  function  in  the  vehicle  (person  or 
mechanism)  transporting  the  information,  will  be  important 
factors  in  determining  how  fast  the  idea,  concept,  or 
device  gets  adopted. 

8.  Perceived  Reward  to  the  Receiver.  As  Lingwood  and  Morris 
(1976,  page  121)  commented,  "...  rewards  are  the  glue 
which  holds  organizations  together  and  provides  the 
response  to  individual  needs  for  recognition  of 
accomplishments  .  .  .  ".  Conventional  wisdom  reminds  us 
that  how  the  rewards  structure  of  an  organization  is 
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perceived  by  an  individual  will  have  a  great  impact  on 
idea  flow  and  the  adoption  of  innovations.  .  . 

S.  Willingness  to  be  Helped.  Willingness  relates  to  the 
individual's  ability  and/or  desire  to  accept  change. 
Awareness,  even  first-hand  knowledge  of  a  new  and/or 
innovative  idea,  is  not  sufficient  to  ensure  its  use. 
There  must  be  a  willingness  and  interest  or,  perhaps  more 
significantly,  an  internal  motivation  to  utilize  a  better 
method,  process,  or  device. 


A  review  of  this  model  and  the  series  of  studies  related  to  it  proved 
useful  because  of  the  close  conceptual  relationship  to  prior  training  device 
acceptance  research,  and  because  the  Navy  organizational  elements  which  were 
studied  or  used  as  a  backdrop  for  the  Naval  Postgraduate  School 's 
investigations  of  technology  transfer  (e.g.,  those  related  to  civil 
engineering)  were  in  general  different  from  those  on  which  we  have  focused. 
This  provided  a  broadened  perspective  from  which  we  were  better  able  to  look 
for  general izability  in  the  course  of  the  present  research. 


ORGANIZATIONAL  DEVELOPMENT 


The  literature  on  organizational  development  is  concerned  with  improving 
organizational  effectiveness  through  a  variety  of  strategies  and  techniques 
directed  at  inducing  change  in  such  areas  as:  job  attitudes,  worker 
motivation,  leadership,  management  style,  quality  of  working  life,  information 
systems;  decision  making  practices;  social  environments;  and  factors 
influencing  resistance  to  change.  (Bowers  and  Franklin,  1973;  Evans,  1967.; 
Applewhite,  1965;  Fredericko,  Brun  and  McCalla,  1980;  Cummings,  1980; 
Faucheux,  Amado  and  Laurent,  1982;  and  Mitchell,  1979). 
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A  comprehensive  review  of  this  very  large  literature  was  outside  the  scope 
of  this  project.  Although  it  is  likely  that  all  of  the  above  factors 
influence  the  working  effectiveness  of  Navy  organizations,  we  concerned 
ourselves  with  a  subset  of  this  literature  which  appeared  to  be  particularly 
relevent  to  organizational  processes  associated  with  training  device  design 
and  innovation.  Some  recent  efforts  to  relate  general  systems  theory  (GST)  to 
organizational  development  (Cummings,  1980;  Lundberg,  I960)  seemed  especially 
helpful.  As  noted  by  Lundberg,  GST,  which  is  currently  popular  in  the  field 
of  management,  is  useful  in  fields  whose  phenomena  are  not  easily  explained  by 
mechanical  or  probablistic  approaches.  Organizations  are  viewed  as  open 
systems.  In  contrast  to  "closed"  systems  of  interrelated  parts  which  function 
within  themselves  and  independently  of  their  context,  "open  systems,"  are 
comprised  of  interdependent  parts  which  accept  from,  respond  to,  and  export 
matter,  information,  and  energy  to  their  environment.  In  addition,  a  system 
may  be  "adaptive",  where  the  parts  exhibit  an  ordered  pattern  of  activity 
which  is  congruent  with  certain  system  ends.  Open,  adaptive  systems  have  a 
dynamic  structure  and  not  only  exchange  matter  and  energy  with  their 
environment,  but  also  information  which  may  serve  as  feedback  (Lundberg, 
1980).  We  view  the  Navy's  training  device  development  and  procurement 
"organization"  as  an  open  adaptive  system,  even  though  it  may  be  a  loosely 
coupled  one. 

Several  concepts  from  GST  that  appear  particularly  appropriate  will  be 
discussed.  First  however,  it  is  necessary  to  observe  that  whether  the  Navy 
has  £n  organization  reponsible  for  training  device  design  and  development  can 
be  questioned.  An  examination  of  the  acquisition  process  (see  Section  3) 


reveals  that  the  Navy  training  device  organization  can  better  be  described  as 
a  loose  affiliation  of  several  sub-organizations  with  more  or  less  separate 
responsibilities  for  need  identification,  funding,  conceptualization,  design, 
development,  construction,  test,  use,  and  support.  Further,  the  particular 
mix  of  Navy  organizations  involved  in  these  activities  depends  partly  on 
whether  the  device  is  being  procured  for  the  air,  surface,  or  submarine 
components  of  the  Navy  and  partly  on  the  initial  stimulus  for  the  device 

( i . e . ,  major  weapon  system  development  vs.  myriad  other  training  requirements 
that  may  be  identified  by  any  fleet  activity  or  Navy  command).  The  degree  to 
which  various  Navy  agencies  get  involved  in  various  phases  of  the  acquisition 
process  can  be  almost  unique  to  each  training  device.  Indeed,  the  "first 
article  trainer"  is  governed  by  a  different  set  of  procurement  rules  (those 
for  R,D,T  8E)  than  are  follow-on  trainers.  This  has  significant  implications 
concerning  the  responsibilities  and  levels  of  authority  that  various  Navy 
sub-organizations  have,  regardless  of  their  eventual  stake  in  trainer  use  and 
effectiveness.  In  the  view  of  some  it  argues  for  substantial  change  in  the 

way  responsibilities  are  assigned  during  the  early  phases  of  the  acquisition 
process.  (Nutter  and  Terrell,  1S82.  See  also  Appendix  C). 

A  related  general  observation  is  that  not  all  of  the  organizations 

involved  in  training  device  procurement  have  the  same  goals  and  reward 
structure.  While  all  involved  parties  might  agree  that  the  objective  is  to 
develop  the  trainers  that  will  deliver  the  most  effective  device  for  meeting 
fleet  readiness  requirements,  it  is  not  at  all  clear  that  the  rewards 
associated  with  this  long-term  objective  are  the  most  compelling  ones 
considering  the  feedback  loops  that  appear  to  govern  shorter-term 
organizational  behavior.  In  fact,  it  seems  likely  that  shorter  term 
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objectives  associated  with  time  and  budget  constraints  may  have  a  more 
profound  impact  on  organisational  behavior  than  does  the  presumed  ultimate 
objective  of  producing  effective  training  devices.  In  this  respect,  the 
Navy's  training  device  acquisition  organization  is  very  different  from 
industrial  organizations.  The  rewards  to  be  obtained  are  only  remotely 
associated  with  customer  satisfaction  in  the  market  place,  and  that  market 
place  is  non-competitive.  Indeed,  many  of  the  players  who  are  involved  early 
in  the  design  and  development  process  may  have  entirely  new  sets  of 
responsibilities  and  interests  by  the  time  the  product  is  delivered.  This,  we 
hypothesize,  has  pronounced  effects  on  the  behavior  of  the  Navy's  training 
device  acquisition  organization  and  it  is  in  this  sense  that  some  of  the  GST 
concepts  appear  to  be  helpful. 

It  is  a  characteristic  of  open  adaptive  systems  that,  over  time, 
considerable  autonomy  of  certain  components  of  the  system  may  develop,  even 
when  there  is  intercomponent  interaction  (Lundberg,  1980).  This  is  the  result 
of  "codes"  developed  in  the  system  as  to  what  constitutes  appropriate  feedback 
information  (Lundberg,  1980).  Feedback  codes  that  relate  directly  to  desired 
system  states  control  the  system,  and  in  this  sense  the  system  actions  are 
goal  directed.  Some  components  of  the  system,  because  they  design  and  monitor 
feedback,  become  relatively  influential.  In  this  regard,  Lundberg  cites 
"Pareto's  Law"  to  the  effect  that  a  relatively  small  percentage  of  inputs 
creates  a  relatively  large  percentage  of  outcomes.  This  seems  to  us  to  apply 
in  the  case  of  many  Navy  training  device  developments,  and  the  model  we  have 
developed  expressly  takes  this  potential  problem  into  account. 


31 


Feedback  information  is  a  pivotal  idea  in  general  systems  theory  and,  it 
is  also  central  to  the  model  of  training  device  development  and  acceptance 
developed  during  this  study.  In  the  systems  context,  the  actual  performance 
of  a  system  (the  training  device  organization)  is  compared  with  a 
predetermined  desired  state  and  information  about  any  deviation  between  the 
two  directs  the  system  into  closer  correspondence.  Feedback  which  counteracts 
deviation  is  called  negative  feedback.  In  contrast,  positive  feedback  is 
information  about  system  actions  that  increases  the  deviation  of  the  system 
from  its  predetermined  goal  state  (Lundberg,  1S80).  Organizational  behavior 
can  change  through  the  application  of  either  negative  or  positive  feedback. 
In  fact,  Lundberg  points  out  that  planned  change  relies  on  negative  feedback, 
which  serves  to  reduce  deviations  from  existing  goals  established  by  managers 
who  presumably  know  best.  But  organizational  development  that  is,  in 
Lundberg's  terms,  moving  an  organization  into  unanticipated  new  environments 
or  states,  or  causing  it  to  reorganize  its  components  for  better  goal  seeking 
requires  positive  feedback  as  well.  Conceptually,  these  notions  appear  to 
relate  to  our  concern  with  how  innovative  technology  and  the  outputs  of 
research  actually  find  their  way  into  the  design  of  new  training  equipment. 
It  seems  certain  that  most  military  organizations  operate  primarily  on 
negative  feedback  and  we  certainly  see  the  Navy's  training  device  acquisition 
process  as  being  dominated  by  management  actions  aimed  at  keeping  system 
behavior  in  close  correspondence  with  preconceived  states.  If  acceptance  of 
innovation  implies  deviation  of  the  organization  from  its  predetermined  goal 
states  and  traditional  reward  structure,  then  hard  sledding  for  innovative 
technology  and  new  concepts  is  certainly  to  be  predicted. 
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Another  important  concept  of  GST  in  the  context  of  organizational 
development  is  that  the  target  of  change  is  less  often  the  people  in  the 

system  whose  attitudes,  leadership  styles,  or  whatever  may  benefit  by  change, 
but  rather  the  targets  are  system  processes.  Included  in  these  are 
informational  processes  which,  Lundberg  (1980)  feels,  may  be  more  fruitful 
targets  than  the  individuals  themselves.  Thus  the  creation,  utilization  and 
extinction  of  various  feedback  loops  become  the  focal  point  for  managing  both 
change  and  development.  Intervention  processes  are  now  seen  as  involving  the 
alteration  of  feedback  loops  both  within  and  between  various  system 

components.  The  creation,  utilization,  and  extinction  of  feedback  loops  is 
seen  as  the  means  by  which  both  progressive  change  and  real  development  take 
place.  Most  interventions  are  seen  by  Lundberg  as  involving  negative  feedback 
loops.  But,  as  noted,  the  theory  suggests  that  positive  feedback  is  a 

necessary  condition  to  drive  the  organization  toward  an  alternative  state.  Of 
course,  even  organizations  that  are  in  the  process  of  developmental  change 
maintain  many  negative  feedback  loops  or  else  the  organization  would  go  out  of 
control . 

Most  organizations  in  the  modern  world  have  multiple  goals,  some  of  which 
may  not  be  clear.  In  addition,  as  Lundberg  (1580)  notes,  most  organizations 
are  not  specified  completely.  Certainly,  we  view  these  as  two  characteristics 
of  the  Navy's  complex  organizational  structure  for  acquiring  training 

systems.  This  suggests  that  some  components  and  processes  will  not  be  under 
feedback  control.  One  of  the  problems  of  a  managed,  planned  change  strategy 
is  that  it  may  thus  deal  with  only  the  tip  of  the  organizational  iceberg 
(Lundberg,  1980). 
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The  system-component  relationships  in  an  organization  differ  in  degree  of 
coupling  or  amount  of  dependence  and  interaction.  Further,  organizational 
boundaries  vary  in  permeability.  We  see  both  of  these  characteristics  in  Navy 
training  device  acquisition.  The  degree  of  organizational  coupling  is  a 
function  of  information  flow  which  of  course  involves  many  feedback  loops. 
Our  model  attempts  to  reflect  these  characteristics. 

Lundberg  quotes  Argyris  (1970)  to  the  effect  that  the  generation  and  use 
of  valid  data  about  organizational  functioning  is  the  central  defining 
characteristic  of  organizational  development.  Trying  to  describe  the 
functioning  of  the  Navy's  training  device  acquisition  "organization,"  and  how 
it  relates  to  innovation  and  change  has  been  a  major  objective  of  the  present 
project  and  it  has  turned  out  to  be  a  far  more  complex  matter  than  we 
originally  envisioned.  As  noted  earlier,  there  is  not  just  one  functioning 
organization— there  are  many.  This  will  be  discussed  in  detail  in  Section  3. 

REVIEW  OF  OFFICIAL  DIRECTIVES  AND  INSTRUCTIONS 


There  are  many  official  directives  and  instructions  bearing  on  the 
training  system  acquisition  process.  The  titles  of  those  we  have  reviewed  are 
given  in  Appendix  A  to  give  the  reader  a  feeling  for  the  subject  matter. 
These  directives  and  instructions  represent  a  substantial  body  of  official 
policy,  and  it  might  be  thought  that  review  of  these  materials  would  reveal 
the  Navy's  training  system  design,  development,  procurement,  introduction,  and 
use  processes.  However,  it  was  clear  from  our  interviews  with  various  members 
of  the  Navy's  training  device  acquisition  organization  that  a  clear 
understanding  of  how  things  are  "really"  done  can't  be  gained  by  reading  these 
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directives  and  instructions.  Certain  directives  and  instructions  represent  a 
codification  of  processes  that  are  well  established  and  working  smoothly  and 
uniformly.  Others  represent  idealized  procedures  which  may  differ 
substantially  from  actual  practice.  In  every  case,  it  has  been  our  experience 
that  personal  interviews  with  people  involved  in  particular  processes  were 
necessary  to  gain  an  understanding  of  current  practice  and  future  directions. 
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SECTION  THREE 


OVERVIEW  OF  THE  ACQUISITION  PROCESS 

During  this  project,  we  conducted  51  meetings  in  which  we  interviewed  76 
persons  from  OPNAV,  NAVSEASYSCOM,  NAVAIRSYSCOM,  NAVEDTRACOM,  NAVTRAEQUIPCEN, 
ASWTRACENPAC ,  SURFPAC,  SUBTRAFAC,  ASWINGSPAC,  SUBGRU5,  and  members  of  Fleet 
Project  Teams  representing  25  different  training  devices  in  various  stages  of 
development  ranging  from  pre-contract  to  obsolescence.  Some  additional  detail 
regarding  these  interviews  is  given  in  Appendix  B. 

The  intent  of  these  interviews  was  to  obtain  the  personal  observations  of 
people  who  are  working  directly  in  training  system  research,  development, 
acquisition,  management,  and/or  use.  We  experienced  a  very  high  degree  of 
cooperation,  and  found  that  most  interviewees  had  well  formed— and  often  very 
strong— attitudes  and  beliefs  regarding  factors  that  influence  training  system 
design,  innovation  acceptance  and  use.  The  results,  while  not  quantitative, 
represent  an  important  body  of  data  concerning  the  organizational  processes 
i nvol ved. 

In  this  section,  we  present  some  of  what  was  learned  from  these  interviews 
in  the  form  of  an  overview  of  the  training  system  acquisition  process  that 
attempts  to  identify  most  of  the  important  players.  We  necessarily  must 
present  a  relatively  generalized  description,  because  the  detailed 
organizational  interactions  are  complex,  often  subtle,  and  vary  considerably 
among  trainers  depending  upon  a  supri singly  large  number  of  unique  factors. 
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Then,  in  Section  4,  a  preliminary  model  of  trainer  utilization  and  innovation 
acceptance  is  presented,  and  in  Section  5,  many  of  the  more  important 
interview  results  are  presented  in  the  form  of  quotations  (unattributed,  of 
course)  which  are  examined  in  the  context  of  the  model. 

STIMULI  FOR  ACQUISITION  OF  TRAINING  SYSTEMS 


The  three  major  reasons  for  acquiring  new  training  systems  are  1)  to 
provide  initial  training  capability  for  new  weapons  systems  2)  to  provide 
improved  or  expanded  training  capability  and  3)  to  provide  a  vehicle  for 
training  technology  research  and  development.  The  latter  two  stimuli 
sometimes  go  hand  in  hand;  certainly  they  can  support  one  another. 

New  training  systems  arising  in  response  to  the  first  need  mentioned 
above,  to  provide  training  capability  in  support  of  any  training  objective, 
for  a  new  weapon  system,  generally  are  subject  to  a  set  of  constraints  which 
are  quite  different  from  those  associated  with  training  systems  arising  from 
the  second  two  stimuli.  This  is  a  consequence  of  the  training  system  being 
"driven"  by  a  weapon  system  acquisition.  Generally,  a  new  weapon  system 
acquisition  is  a  very  expensive  and  visible  enterprise,  compared  to  typical 
training  system  acquisitions.  Training  system  acquisition  which  is  associated 
with  a  major  weapon  system  acquisition  can  sometimes  benefit  from  the  high 
priority  and  substantial  "power"  which  typically  accrues  to  the  latter; 
however,  the  training  system  acquisition  often  is  also  subjected  to 
substantial  pressure,  usually  in  the  form  of  time  and  budgetary  constraints, 
because  of  its  subordination  to  the  weapon  system  project. 
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ORGANIZATIONAL  RELATIONSHIPS 


Figure  3  shows  tentative  "organizational  envelopes"  that  identify  major 
players  and  relationships  in  training  system  acquisition  and  use.  These 
players  and  relationships  will  be  discussed  in  turn  below. 


RDT&E,N  Fund  Flow 


Research  and  development  in  the  Navy  is  funded  under  the  "Research, 
Development,  Test,  and  Engineering,  Navy"  (RDT&E.N)  account.  Since  August, 
1977  first-article  trainers  must  also  be  funded  under  the  RDT&E,N  account, 
rather  than  procurement  accounts,  in  order  to  create  better  control  and 
systematic  development.  This  requirement  imposes  significant  constraints  on 
first-article  development  and  on  follow-on  trainers.  Consequently,  the 
offices  which  monitor  and  control  RDT&E,N  funds  play  an  important  role,  not 
only  in  more  "basic"  research  and  training  system  innovation,  but  in 
first-article  operational  trainer  development  as  well. 

The  offices  concerned  with  RDT&E,N  funds  are  shown  in  the  top  line  of 
blocks  in  Figure  3.  Most  of  these  offices  have  some  impact  on  all  R&D  and 
first-article  trainers,  and  substantial  impact  on  selected 
trainers.  The  offices  include:  The  Office  of  the  Undersecretary  of  Defense 
for  Research  and  Engineering;  The  Office  of  the  Comptroller  of  the  Navy;  the 
Assistant  Secretary  of  the  Navy  for  Research,  Engineering,  and  Systems;  The 
Office  of  Naval  Research;  and  the  OPNAV  office  of  Research,  Development,  Test, 
and  Engineering  (OP-098).  These  offices  have  a  very  substantial  and  active 
influence  on  training  system  innovation,  and  informal  communication  among  key 
members  of  these  offices  and  with  some  of  the  other  players  shown  in  Figure  3 
is  a  very  important  factor. 
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Office  of  the  Chief  of  Naval  Operations 


All  training  system  acquisition  projects  must  have  appropriate  sponsors 
within  The  Office  of  the  Chief  of  Naval  Operations  (OPNAV).  OPNAV  resource 
sponsors  include  0P-01,  0P-02,  0P-03,  and  0P-05.  Each  will  be  discussed  in 
turn. 

0P-01:  Deputy  Chief  of  Naval  Operations,  Manpower,  Personnel,  and 

Training.  Within  0P-01,  the  current  Research,  Development  and  Studies  Branch 
(Code  OP-115)  deals  with  trainers.  OP-115  sponsors  training  technology 
research  and  development  (6.3  funding),  and  interacts  with  and  sponsors  CNET 
Code  N-5  and  NAVTRAEQUIPCEN  Code  N-7,  which  are  represented  in  Figure  3. 
OP-115  maintains  an  awareness  of  Office  of  Naval  Research  6.1  and  6.2  programs 
in  order  to  plan  for  appropriate  6.3  programs.  OP-115  also  sponsors  some  6.4 
training  developments  where  these  developments  are  generic  and/or  do  not 
clearly  fall  to  a  particular  warfare  area  sponsor  (i.e.,  0P-02,  0P-03,  or 
0P-05). 

0P-02:  Deputy  Chief  of  Naval  Operations,  Submarine  Warfare.  Within 

0P-02,  Manpower  and  Training  Requirements  Division  (Code  0P-2S)  sponsors 
trainers  for  submarine  systems.  0P-2S  tends  to  buy  trainers  as  parts  of  prime 
weapon  system  acquisition,  assigning  responsbil ity  for  the  aquisition  trainers 
to  Naval  Sea  Systems  Command. 

0P-03:  Deputy  Chief  of  Naval  Operations,  Surface  Warfare.  The  Manpower 
and  Training  Requirements  Division,  OP-39,  sponsors  trainers  in  the  surface 
warfare  area.  OP-39  typically  maintains  close  liaison  with  Naval  Training 
Equipment  Center  (NAVTRAEQUIPCEN)  regarding  surface  trainer  development. 


OP-39 
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frequently  buys  trainers  as  commodities  separate  from  weapons  systems,  using 
NAVTRAEQUIPCEN  as  the  developing  agency. 


0P-05:  Deputy  Chief  of  Naval  Operations,  Air  Warfare.  The  Aviation 

Manpower  and  Training  Division,  OP-59,  sponsors  air  trainers.  OP-59 
frequently  relies  heavily  on  Naval  Air  Systems  Command's  Weapons  Training 
Division  (AIR-413)  for  trainer  requirements,  design,  and  acquisition.  0P-05 
tends  to  buy  trainers  as  part  of  the  prime  weapons  system  acquisition. 

Naval  Material  Command 


The  Naval  Material  Command  (NAVMAT)  provides  for  the  total  system  and 

material  support  needs  of  the  operating  forces  of  the  Navy  for  equipment, 

weapons  and  weapons  systems,  materials,  supplies,  facilities,  maintenance,  and 
supporting  services.  (U.S.  Government  Manual,  1978).  As  a  "training  support 
agent"  (NAVCOMPT  Manual  075148),  the  Chief  of  Naval  Material  provides 
initial  training,  equipment,  and  materials,  often  via  Naval  Training 

Equipment  Center,  which  is  under  the  command  of  the  Chief  of  Naval  Education 
and  Training,  but  which  is  assigned  for  additional  duty  to  the  Chief  of  Naval 
Material  and  others  (CNETINST  5450.8/NAVMATINST  5450.28).  As  may  be  seen  in 
Figure  3,  the  Chief  of  Naval  Material  delegates  authority  to  fulfil  his 

training  support  agent  role  to  the  Naval  Sea  Systems  Command  (NAVSEA)  and 
Naval  Air  Systems  Command  (NAVAIR). 


^"Initial  training"  is  that  training  performed  by  the  Training  Support  Agent 
[eg.,  NAVMAT]  pending  the  opportunity  for  the  Training  Agent  [eg.,  CNETj  to 
acquire  the  capability  for  training  (OPNAVINST  1500. 8J).  The  training  which 
the  Training  Agent  performs  is  referred  to  as  "follow-on  training." 


41 


Naval  aviation  has  a  lengthy  history  of  utilizing  training  devices,  to  a 
greater  degree  than  in  the  surface  and  submarine  warfare  areas.  Consequently, 
Naval  Air  Systems  Command,  both  by  custom  and  the  development  of  technical 
expertise,  often  exercises  a  substantial  degree  of  autonomy  in  developing 
training  devices  for  naval  aviation.  This  is  reflected  in  Figure  3,  which 
shows  a  direct  line  of  training  system  support  from  NAVAIR  to  the  Air  Type 
Commanders  ( i . e . ,  COMNAVAJRPAC  and  COMNAVAIRLANT) ,  with  the  a  dashed  line 
representing  "technical  assistance"  by  NAVTRAEQUIPCEN.  AIR-413,  Weapons 
Training  Division,  is  responsible  for  trainer  acquisition  (6.4  funding). 
AIR-340,  Equipment  and  Support,  is  responsible  for  trainer  technology  programs 
(6.3  funding). 

Naval  Education  and  Training  Command 


The  Chief  of  Naval  Education  and  Training  (CNET),  under  the  Chief  of  Naval 
Operations,  is  responsible  for  assigned  shore-based  education  and  training  of 
Navy  personnel;  develops  specifically  designated  education  and  training  afloat 
programs  for  the  fleets,  participates  with  research  and  development  activities 
in  the  development  and  implementation  of  teaching/training  systems  and  devices 
for  optimal  education  and  training;  and  executes  a  number  of  other  Navy 
education  and  training  responsibilities.  CNET,  as  a  training  agent,  provides 
follow-on  surface  and  submarine  trainers  and  training  via  Naval  Training 
Equipment  Center  (NAVTRAEQUIPCEN)  and  various  functional  commanders  (eg., 
COMTRAPAC,  COMTRALANT,  CNTECHTRA) ,  as  discussed  below. 

Naval  Training  Equipment  Center  (NAVTRAEQUIPCEN).  NAVTRAEQUIPCEN  reports 
to  CNET,  but  has  additional  duty  tasking  to  NAVMAT,  indicated  in  Figure  3  by 
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the  dashed  NAVMAT  organizational  boundary  line  enveloping  NAVTRAEQUIPCEN. 
NAVTRAEQUIPCEN  is  recognized  as  the  Navy's  primary  technical  agent  for 
training  devices  (Nutter  and  Terrell,  1S82).  Specific  functions  and 
responsibilities  of  NAVTRAEQUIPCEN  as  derived  from  CNETINST  5450.31a  and  other 
directives,  include: 

o  perform  functions  in  support  of  CNET,  CHNAVMAT,  SYSCOMs,  PMs,  and  other 
activities  in  the  design,  development,  acquisition,  and  logistics 
support  of  training  devices  and  equipment 

o  identify  resource  requirements  for  training  device  initial  spares, 
modification,  and  factory  training 

o  assist  CNET  and  functional  commanders  in  initial  analysis  of  training 
device  needs 

o  perform  technical  assessment  of  training  device  feasibility,  cost, 
approaches  and  alternatives 

o  provide  facility,  personnel  and  OPTAR  requirement  criteria  to 
functional  command  training  activity 

o  participate  in  Submarine  Trainer  Working  Group  and  Surface  Warfare 
Trainer  Group  functions 

o  perform  training  device  engeineering  development  functions  including 
front-end  analysis,  planning,  engineering,  logistics,  and  contracting 

o  perform  equipment  test  and  evaluation 

o  conduct  training  research  (in-house  and  contract) 

o  establish  functional  baseline 

o  develop  Program  Master  Plan 

o  develop  POM  justification  data 

CNET  Functional  Commands  and  Schools.  CNET's  functional  commanders 
include  the  Chief  of  Naval  Technical  Training  (CNTECHTRA),  Chief  of  Naval 
Aviation  Training  (CNATRA);  Commander,  Training  Command,  U.S.  Atlantic  Fleet 
( COMTRALANT ) ;  and  Commander,  Training  Command,  U.S.  Pacific  Fleet 
(COMTRAPAC).  They  are  responsible  to  CNET  for  conduct  of  the  training  and 
identification  of  all  resource  requirements  to  support  that  training.  In 
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addition,  training  curriculum  control  authority  resides  within  the  functional 
command.  The  functional  commanders  have  numerous  training  activities  or 
schools  which  are  responsible  to  them  and  which  support  training  directly.  It 
is  in  these  schools  that  training  devices,  instructors,  and  students  come 
together,  comprising  the  ultimate  "point  of  training."  As  shown  in  Figure  3, 
the  major  training  support  for  the  submarine  and  surface  type  commanders 
(COMSUBPAC/LANT,  COMNAVSURFPAC/LANT )  is  provided  by  CNET's  schools,  which  in 
some  cases  have  additional  duty  tasking  to  the  type  commanders,  as  shown  by 
the  dashed  line  of  Figure  3  which  envelopes  the  schools. 


However,  the  majority  of  advanced  training  support  for  the  air  type 
commanders  (COMNAVAIRPAC/LANT)  is  not  provided  by  CNET's  schools;  the  air  type 
commanders  provide  for  most  of  their  own  training  organically  ( i e . ,  within 
their  own  commands,  by  the  Fleet  Aviation  Specialized  Operational  Training 
Groups  (FASO's),  the  Fleet  Replacement  Squadrons  (FDS's)  and  Fleet  Squadrons 
(FS's),  discussed  later),  with  close  support  from  Naval  Air  Systems  Command, 
as  shown  in  Figure  3.  Frequently,  NAVTRAEQUIPCEN  offers  direct  support  to  the 
Air  type  commanders  as  well,  particularly  where  NAVTRAEQUIPCEN  was  the  trainer 
developing  agent. 


CNET  R&D.  CNET  has  a  substantial  R&D  responsibility,  and  a  substantial 
responsibility  in  facilitating  the  transfer  of  RDT&E  results  from  one  funding 
category  to  another.  As  Nutter  and  Terrell  (1982)  observed 


CNET  N-5  (Research,  Development,  Test  and  Evaluation)  has  cognizance  of 
6.1,  6.2,  and  6.3,  the  technology  based  development  categories;  and  CNET 
N-9,  Training  System  Management,  has  cognizance  over  6.4,  the  hardware 
based  operational  capabilities  development  category  for  training  devices. 
CNET  N-9  is  also  responsible  for  the  planning  and  programming  for 
associated  resources  in  other  appropriation  categories  ( i . e . ,  MPN,  OM&N, 
and  MILCON).  It  is  obviously  important  to  maintain  good  communications 
between  N-5  and  N-S  to  ensure  the  flow  of  technology  base  information  from 
technology  development  to  engineering,  and  technology  requirements  from 
engineering  to  technology  base  development. 
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NAVTRAEQUIPCEN  also  conducts  a  substantial  research  program,  and  the  comment 
above  regarding  coordination  between  CNET  codes  N-5  and  N-S  certainly  extends 
to  the  NAVTRAEQUIPCEN  N-7  research  codes  as  well. 

Quality  Assurance  and  Revalidation.  CNET  has  a  Quality  Assurance  and 
Revalidation  (QA&R)  program  which  inspects  all  major  training  devices  in  the 
field  periodically  with  respect  to  material  reliability,  utilization, 
configuration  management,  achievement  of  training  objectives,  and  other 
factors.  The  QA&R  program  is  intended  to  be  responsive  to  the  needs  of  the 
users  (i.e.,  the  type  commanders  and  the  training  commands)  and  frequency  of 
inspection  is  based  on  inputs  from  them.  The  results  of  QA&R  inspections  are 
sent  to  these  users  and  also  to  CNET,  NAVTRAEQUIPCEN,  the  appropriate  Systems 
Command  and  others.  The  QA&R  program  provides  a  formal  method  for  evaluating 
and  feeding  back  user  complaints  and  trainer  and/or  support  deficiencies. 

The  Fleet 


The  Fleet  Commanders-in-Chief  (CINCPACFLT,  CINCLANTFLT)  rely  substantially 
on  the  Chief  of  Naval  Material  (CHNAVMAT)  and  the  Chief  of  Naval  Education  and 
Training  (CNET)  for  initial  and  follow-on  training.  Figure  3  indicates  some 
of  the  relationships  involved.  The  submarine  type  commanders  (COMSUBPAC, 
COMSUBLANT)  and  the  surface  type  commanders  (COMNAYSURFPAC,  COMNAVSURFLANT) 
make  extensive  use  of  training  ashore  provided  by  CNET's  functional  commanders 
(CNTECHRA,  COMTRAPAC ,  COMTRALANT,  etc.).  The  air  type  commanders 
(COMNAVAIRPAC,  COMNAVAIRLANT )  provide  much  more  of  their  training  internally. 
The  FASO's  are  the  custodians  and  maintainers  of  AIRPAC/LANT  trainers.  The 
FRS's  provide  replacement  training  using  the  trainers,  and  focus  on  safety  and 
training  in  fundamentals.  The  FS's  use  the  trainers  to  provide  advanced 
trai ni ng. 
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Surface  Warfare  Trainer  Group  (SWTG)  and  Submarine  Traini ng/Trai ner  Working 

Group  (STTWG) 


These  working  groups  were  established  by  the  Chief  of  Naval  Operations  to 
provide  and  promote  communications  and  to  provide  a  forum  for  identification 
and  resolution  of  major  trainer  problems  and  issues.  The  working  groups 
generally  meet  twice  a  year  and  involve  a  wide  spectrum  of  members, 
representing  OPNAV,  SYSCOM's,  CNET,  various  schools,  the  fleet  and  Naval 
Training  Equipment  Center.  The  working  groups  identify  trainer  needs, 
consider  alternatives,  prioritize  trainer  projects,  and  make  recommendations 
concerning  Fleet  Project  Teams.  The  working  groups  provide  an  important  forum 
for  the  exchange  of  information  and  they  also  serve  to  enhance  innovation 
acceptance  by  providing  for  user  involvement. 

Fleet  Project  Teams 


The  Fleet  Project  Team  concept  is  officially  instituted  in  OPNAV 
Instruction  1551.7,  "Fleet  Participation  in  Development,  Acquisition,  and 
Acceptance  of  Major  Training  Devices."  In  that  instruction,  the  Fleet  Project 
Team  is  defined  to  be  "a  group  of  knowledgeable  representatives  from  the  fleet 
or  other  user  and  interested  non-user  activities,  consisting  of  qualified 
military  and/or  civilian  personnel  designated  by  cognizant  commands.  The  FPT 
will  assist  and  advise  the  Training  Device  Development  and  Acquisition 
Activity  in  development,  acquisition,  and  acceptance  of  specifically  designed 
training  devices."  A  copy  of  this  instruction  is  provided  in  Appendix  C,  and 
the  reader  will  see  from  the  definition  of  the  Fleet  Project  Team's  role. 
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functions,  and  duties,  that  it  may  play  a  very  important  role  in  training 


device  development,  acquisition,  and  introduction,  depending  on  how  many  of 
the  duties  listed  are  actually  assigned  to  the  FPT  and  whether  the  FPT  has 
available  to  it  the  resources  to  properly  discharge  these  duties. 

Contractors 


In  most  cases,  the  Training  Device  Development/Acquisition  Agent  (e.g.. 
Naval  Training  Equipment  Center,  Naval  Air  Systems  Command,  etc.)  will 
contract  with  a  private-sector  organization  for  the  actual  construction  of  a 
training  device.  Sometimes,  the  prime  contract  for  a  weapon  system  will 
include  development  and  delivery  of  a  supporting  trainer  or  trainers  and 
training  materials.  In  any  event,  a  contractor  almost  always  plays  a  major 
role  in  the  final  realization  of  a  training  device.  Since  a  training  device 
is  rarely  specified  to  minute  detail  in  a  Request  for  Proposal  (RFP),  the 
contractor  often  has  considerable  opportunity  to  be  innovative  in  his  design 
of  the  desired  product.  Because  of  this  freedom,  the  Navy  realizes  certain 

benefits.  For  example,  much  of  the  innovation  in  training  technology  which 

finds  its  way  into  Navy  training  devices  comes  from  trainer  contractors  as  a 
consequence  of  the  competetive  procurement  process,  which  encourages  technical 
creativity  and  rewards  it.  On  the  other  hand,  there  are  some  negative 

aspects:  There  is  considerable  variability  among  contractors '  abilities  to 

perform  on-time,  within  budget,  and  to  a  high  standard  of  technical  quality. 
Unfortunately,  it  is  not  always  possible  to  predict  how  well  a  contractor  will 
perform.  Consequently,  contractors  introduce  considerable  variability  into 
the  acquisition  process. 
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Navy  Laboratories 


A  number  of  Navy  laboratories  contribute  to  innovation  in  training 
devices.  Among  these  are  Naval  Undersea  System  Center  (Newport  and  New 
London),  Naval  Ocean  Systems  Center,  Naval  Personnel  Research  and  Development 
Center,  and  Naval  Air  Development  Center.  The  role  which  Navy  Labs  play,  if 
any,  with  regard  to  a  specific  trainer  is  unique  for  that  trainer.  It  may  be 
major,  or  of  little  consequence. 

CRITICAL  IMPACT  OF  TIME  CONSTRAINTS 


Pervasive  throughout  the  acquisition  cycle  of  trainers  intended  to  support 
new  weapon  systems  is  the  race  to  meet  deadlines.  This  time  pressure  can  have 
adverse  consequences  on  the  systematic  and  orderly  development  of  training 
systems.  As  Nutter  and  Terrell  (1982)  observe: 


Two  critical  time  constraints  are,  from  a  practical  standpoint,  not  likely 
to  be  subject  to  change.  They  are  (1)  that  time  point  in  the  weapon 
system  acquisition  cycle  when  the  training  equipment  must  be  onsite  and 
ready  for  training  (RFT)  and  (2)  the  time  required  to  complete  the 
programming  and  budgeting  cycle.  The  weapon  system  or  equipment  program 
manager  normally  establishes  the  training  equipment  RFT  date.  This  date 
must  be  met  if  trained  personnel  are  to  be  available  to  operate  and 
maintain  the  system  or  equipment  when  delivered  and  if  CNET  is  to  meet 
pipeline  training  requirements.  Failure  to  meet  the  RFT  date  severely 
restricts  the  capability  of  the  operational  forces  to  meet  their  mission 
commitments. 

For  major  training  device  (prototype)  programs,  OPNAVINST  1500.8  states 
that  the  minimum  lead  times  required  to  meet  RFT  dates  are  (1)  5  years  for 
military  construction  projects  (2)  4  years  for  major  training  devices,  and 
(3)  3  years  for  billets  and  expense  dollars.  Historical  evidence  suggests 
a  5-year  phasing  between  POM  input  and  training  device  RFT  for  major 
prototype  RDT&E  devices  to  be  more  realistic.  The  NAVCOMPT  Manual  075148 
states  that  follow-on  devices  can  only  be  programmed  in  years  subsequent 
to  the  successful  demonstration  of  the  prototype  unit.  The  CNO  resource 
sponsor  determines  what  constitutes  a  successful  demonstration.  This  may 
take  place  at  the  prototype  RFT  date  or  1  year  prior  at  design  approval. 
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These  programming  and  budgeting  time  requirements  place  severe  limitations 
on  the  time  available  to  perform  the  various  training  system  design 
analysis  functions  required  prior  to  identifying  the  training  device 
requirement.  Technically,  these  analysis  are  performed  prior  to  the  POM 
submission,  a  period  when  technical  details  relating  to  the  weapon  system 
or  equipment  are  typically  minimal.  From  a  practical  standpoint  then, 
data  necessary  to  complete  all  of  the  stipulated  tasks  prior  to  initiating 
the  training  device  acquisition  process  are  not  available  when  required. 
Therefore,  modified  analyses,  based  on  earliest  information  available  and 
made  progressively  more  definitive  as  the  weapon  system  or  equipment 
program  progresses,  should  be  used  for  initiating  training  equipment 
acquisition  programs  (i.e.,  POM  submittal). 

Another  factor  related  to  front-end  training  analyses  concerns  the 
availability  of  funds  often  required  for  the  conduct  of  such  analyses. 
Due  to  manpower  limitations  and  commitments,  front-end  analyses  cannot 
always  be  conducted  in-hcuse;  contractor  effort  is  sometimes  required  to 
support  front-end  training  analyses  efforts.  The  present  procedure  for 
POM  submission  for  prototype  training  device  acquisition  is  not 
sufficiently  early  to  provide  funds  for  all  required  front-end  training 
analyses . 


Obviously,  compromises  in  systematic  training  system  development  may 
unfavorably  impact  the  likelihood  of  fully  satisfactory  user  acceptance. 


IN  SUMMARY 


The  Navy's  training  system  development  and  acquisition  "organization"  is 
very  complex,  indeed,  and  is  perhaps  best  described  as  a  loosely-coupled 
federation  of  organizations  whose  linkages  vary  on  an  £d  hoc  basis  depending 
on  many  factors. 


In  the  next  section,  a  model  is  presented  which  attempts  to  simplify  and 
generalize  organizational  and  device-specific  factors  which  we  hypothesize  are 
most  important  in  achieving  satisfactory  user  acceptance  of  innovations. 
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SECTION  FOUR 


A  MODEL  FOR  PREDICTING  ORGANIZATIONAL  ACCEPTANCE 
OF  TECHNOLOGICAL  CHANGE 

Based  on  our  review  of  the  literature  and  Navy  directives,  as  well  as  the 
interviews  among  representatives  of  the  organizations  identified  in  Section 
Three,  we  have  developed  a  predictive  model  of  innovation  acceptance.  The 
model  has  a  component  which  is  intended  to  reflect  organizational  factors  and 
a  second  component  to  reflect  specific  features  of  innovations.  The  model  is 
intended  to  be  applicable  to  many  different  kinds  of  organizations;  however, 
we  took  particular  care  to  allow  the  model  to  accomodate  training  technology 
transfer  and  training  innovation  acceptance  processes  as  we  have  observed  them 
in  the  Navy. 

RESEARCH  UTILIZATION  AND  INNOVATION  IN  LARGE  ORGANIZATIONS 


In  large  organizations,  such  as  the  Navy,  a  number  of  distinct 
organizational  entities  may  be  involved  in  the  research  utilization  and 
innovation  process.  Figure  4  presents  a  generalized  organizational  diagram 
depicting  some  important  organizational  units  involved  in  the  process  of 
training  system  innovation  and  user  acceptance.  Each  of  these  units  will  be 
discussed  briefly. 

The  "funding  organization"  represents  an  executive  function  of  approval 
and  funding,  and  of  overall  program  planning,  but  with  relatively  little 
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Figure  4.  Generalized  organizational  diagram 


involvement  in  the  details  of  technical  development.  The  analogue  of  this 

entity  in  the  Navy  is  the  Office  of  the  Chief  of  Naval  Operations. 

The  "developing  organization"  is  the  focal  point  of  the  research 
utilization  and  trainer  innovation  process.  Typically,  the  developing 
organization  anticipates  needs,  gets  projects  officially  recognized  and  funded 
by  the  funding  organization,  develops  the  functional  specifications  of  an 
innovative  device,  contracts  with  a  "builder"  to  produce  a  tangible  product, 
and  delivers  the  product  to  the  user.  Analogues  in  the  Navy  of  this 
organizational  element,  depending  on  a  variety  of  factors,  include  Naval 
Training  Equipment  Center,  the  Systems  Commands,  and  sometimes  the  Navy 

laboratories. 

The  "builder"  is  that  organization  which  implements  the  functional 

specification  to  produce  a  tangible  product.  Most  often,  the  Navy's 
"developing  organizations"  select  and  contract  with  private  sector 
organizations  to  fulfill  the  builder  function,  although  occasionally  the 
function  may  be  performed  by  a  Navy  laboratory  or  other  activity. 

The  "user"  is,  of  course,  the  agency  which  puts  an  innovation  to  practical 
use  and  which,  in  varying  degrees,  accepts  or  rejects  it.  In  our  field 

interviews,  we  concentrated  on  innovations  in  Navy  training  systems,  and  in 
that  connection  found  two  important  organizations  which  together  may  be 
considered  to  comprise  the  user.  The  first  is  the  training  orgnization,  whose 
primary  objective  is  to  train,  and  it  employs  innovations  in  training  systems 
in  accomplishing  that  goal.  In  doing  so,  it  supports  the  other  "user," 
namely,  the  operational  organization.  The  role  of  the  operational 
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organization  is  to  pursue  the  primary  objectives  of  the  enterprise  as  a  whole, 
and  it  is  supported  in  doing  this  by  the  training  organization.  Both  are 

users  of,  and  acceptors/rejectors  of,  training  system  innovations.  Examples 
in  the  Navy  of  the  training  function  include  the  Naval  Education  and  Training 
Command  and  its  various  functional  commands  and  schools.  Examples  of  the 
operational  organization  in  the  Navy  include  the  Atlantic  and  Pacific  Fleets. 

The  "research  organization"  performs  the  function  of  generating  innovative 
knowledge.  Research  organizations  may  exist  as  subunits  of  the  organizational 
elements  discussed  above.  For  example,  in  the  Navy,  a  "developing 
organization"  (Naval  Training  Equipment  Center)  and  a  "training  organization" 
(Naval  Education  and  Training  Command)  have  organic  research  organizations. 
Likewise,  contractors  often  have  in-house  research  organizations.  Frequently, 
though,  even  research  organizations  which  are  organic  to  the  funder, 

developer,  builder,  or  user  nonetheless  retain  a  somewhat  distinctive 
character;  and  sometimes  they  may  be  organizationally  quite  distant  from  the 
other  players. 

ORGANIZATIONAL  INTERACTION  MATRIX 

We  believe  that  any  model  which  attempts  to  deal  successfully  with 
technology  transfer  in  large  organizations  must  not  deal  simply  with  a  "giver" 
of  innovative  information  to  a  "receiver."  Models  which  involve  only  two 

parties  (giver  and  receiver)  are  bound  to  overlook  some  of  the  important 

organizational  interactions  and  necessary  feedback  processes  which  bear  upon 
research  utilization  and  innovation  acceptance. 


N 
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Figure  5  presents  an  organizational  interaction  matrix,  which  depicts 
schematically  the  organizational  interactions  among  units  of  the  type  shown  in 
Figure  4  (i.e.,  researcher,  funder,  developer,  builder,  user).  Each  cell  of 
the  matrix  represents  an  interaction  between  a  player  "X"  (listed  on  the  left 
side  of  the  matrix)  and  a  player  "Y"  (listed  across  the  top  of  the  matrix). 
In  order  to  understand  the  process  of  generating  innovative  knowledge, 
reducing  it  to  practice,  introducing  it  to  users,  and  gaining  acceptance,  we 
believe  it  is  necessary  to  consider  a  variety  of  interaction  factors  for  most, 
if  not  all,  of  the  possible  pairings  of  X  and  Y  depicted  in  Figure  5.  We  will 
turn  our  attention  to  these  interaction  factors. 

ORGANIZATIONAL  INTERACTION  FACTORS 


On  the  basis  of  the  literature  review  and  our  first-hand  observations,  we 
believe  that  the  nine  organizational  interaction  factors  shown  in  Figure  6 
capture  most  of  the  important  facets  of  organizational  interaction  as  it 
impacts  technology  transfer.  The  first  three  factors  (seek,  hear,  read)  bear 
upon  the  ability  and  motivation  (power,  qualification,  skill,  willingness)  of 
a  "player"  to  receive  innovative  information;  factors  4  and  5  (utilize,  get 
reward)  bear  on  the  ability  of  the  player  to  employ  knowledge  gained;  and 
factors  6-9  (talk,  write,  show  credibility,  give  reward)  bear  on  the  ability 
and  motivation  of  a  player  to  transmit  innovative  information  effectively.  We 
will  discuss  briefly  each  of  these  nine  factors  in  turn. 
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Figure  5.  Organizational  interaction  matrix. 


Seek 


This  factor  has  to  do  with  the  willingness  and  motivation  of  a  player  to 
seek  innovation  information  actively.  This  seeking  behavior  is  an  important 
component  of  the  ability  of  a  player  to  serve  in  the  role  of  a  "linker" 
(gate-keeper,  opinion  leader)  in  technology  transfer  theory  (Jolly,  et.  al., 
1978).  Our  interviews  have  included,  for  example,  instances  where  "developer" 
did  not  actively  seek  information  from  "researcher." 

Hear/View  and  Read 


"Hear/view"  and  "read"  are  separated  in  our  model,  because  they  require 
different  degrees  of  effort  and  skill  on  the  part  of  the  receiver,  and  because 
the  oral /demonstration  form  of  communication  has  been  found  to  be  more 
effective  than  the  documentary  mode  in  regard  to  change  advocacy  (Rogers  and 
Shoemaker,  1971).  Motivation  and  ability  to  hear/view  and  to  read  innovation 
information  are  important  aspects  of  the  technology  transfer  and  innovation 
acceptance  process.  We  hypothesize  that  different  groups  (researcher,  funder, 
developer,  builder,  user)  have  different  preferences  between  the 
oral /demonstration  and  the  written  forms  of  communication,  and  that  effective 
interaction  among  groups  can  be  impaired  if  these  preferences  are  not  heeded. 

Utilize 


This  factor  refers  to  the  capacity  of  a  potential  user  of  innovation 
information  to  employ  that  information  in  furthering  the  process  of  change. 
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Ability  to  utilize  information  successfully  depends  on  such  things  as 
motivation,  training,  status  and  a  variety  of  pre-dispositional  variables 
( Jol ly ,  et  al . ,  1 978) . ) . 

Get  Reward 


This  factor  refers  to  the  ability  of  "X"  to  get  reward  (reinforcement, 
punishment)  from  a  player  “Y".  Reinforcement  is  an  important  determinant  of 
individual  behavior  in  regard  to  change  (and  in  general).  Rewards  are 
important  elements  of  the  feedback  loops  postulated  in  general  systems  theory 
to  be  necessary  for  altering  organizational  processes  (Lundberg,  1880). 
However,  not  all  "Y's"  are  able  to  provide  the  same  degree  of  rewards  to  the 
"X's".  For  example,  "developer"  obviously  gets  reinforced  by  "funder,"  but  in 
our  observations  the  degree  of  reward  "developer"  feels  he  can  get  from 
"researcher"  is  at  times  quite  small.  Thus  the  ability  of  each  X  to  get 
reward  from  each  Y  must  be  considered  independently  for  each  of  the  cells  of 
the  organizational  interaction  matrix  shown  in  Figure  5. 

Talk/Demonstrate  and  Write 


These  factors  are  the  "transmitting"  analogues  of  factors  2  and  3, 
"hear/view"  and  "read."  Again,  we  differentiate  between  the  methods  of 
communication  because  of  their  differential  effectiveness  for  various 
purposes,  and  because  the  preferred  method  may  vary  depending  on  which  players 
interact  (i.e.,  depending  on  which  cell  of  the  organizational  interaction 
matrix  one  considers).  For  example,  we  have  observed  that  some  "researchers" 
prefer  to  communicate  to  "developers"  via  written  technical  reports,  which 
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1.  seek  innovation  information 

2.  hear/view  innovation  information 

3.  read  innovation  information 

4.  utilize  innovation  information 

5.  get  reward 

6.  talk/demonstrate  innovation  information 

7.  write  innovation  information 

8.  show  credibility 

9.  give  reward 


✓ 


Figure  6.  Nine  organizational  interaction  factors 


to/ from  "Y" 
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may  not  be  very  effective  when  "developer"  prefers  to  "hear/vi ew"  rather  than 
to  "read." 

Show  Credibility 

Many  investigators  have  found  that  the  credibility  of  a  transmitter  of 
innovation  information  is  an  important  factor  in  the  behavior  of  the  receivers 
of  that  information  (Jolly,  et.  al . ,  1978).  The  ability  of  player  X  to  show 
credibility  to  player  Y  will  vary  according  to  characteristics  of  the 
information  source  and  personal  characteristics  of  players  X  and  Y  (Jolly, 
1980).  We  have  seen  examples  where  "researcher"  has  been  able  to  establish 
credibility  with  "user"  only  by  showing  detailed  knowledge  of  the  user's 
operational  problems. 

Give  Reward 


This  factor  measures  the  ability  of  X  to  give  reward  to  Y,  which  is  to 
say,  to  reinforce  Y’s  behavior  positively  (or  negatively)  in  regard  to  the 
innovation.  Clearly,  as  discussed  under  factor  5  (ability  of  X  to  get  reward 
from  Y),  this  is  an  important  organizational  interaction  factor.  Not  all 
players  in  the  organizational  matrix  are  in  a  position  to  give  reward  to  all 
other  players. 

INTERACTIONAL  ASYMMETRY 


The  interactional  variables  include  those  related 
innovation  information  and  those  related  to  "transmitting" 


to  "receiving" 
information,  as 
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described  above.  Consequently,  it  might  be  assumed  that  after  these  nine 
factors  have  been  evaluated  for  a  given  cell  of  the  organizational  interaction 
matrix  (Figure  5),  the  relationship  betwen  player  X  and  player  Y  would  be 
adequately  characterized,  and  therefore  the  cell  associated  with  the  converse 
relationship  (player  Y  to/from  player  X)  would  be  redundant.  That  is  to  say, 
it  would  be  assumed  that  the  organizational  interaction  matrix  might  be 
symmetrical  (i.e.,  that  cell  =  C.^ )  and  therefore  only  half  the  matrix 

would  be  needed  to  characterize  organizational  interactions. 

However,  organizational  interactions  are  not  necessarily  symmetrical.  For 
example,  player  X  may  be  willing  and  able  to  "hear"  (factor  #2)  player  Y,  but 
player  Y  may  not  be  inclined  to  "hear"  player  X.  Player  Y  may  be  highly 
motivated  to  talk  Player  X,  but  Player  X  may  prefer  to  write  to  Player  Y. 
Therefore,  in  general,  the  complete  matrix  is  needed  to  characterize 
organizational  interactions  adequately. 

We  hypothesize  that  interactional  asymmetry  can  be  an  important  factor  in 
research  utilization  and  innovation  acceptance.  The  process  of  evaluating  the 
nine  interaction  factors  for  each  cell  of  the  orgni zational  interaction  matrix 
should  help  to  make  asymmetrical  relationships  evident. 

ORGANIZATIONAL  COMPONENT  OF  THE  MODEL 


Conceptually,  each  of  the  nine  interaction  factors  shown  in  Figure  6  can 
be  quantified  for  each  of  the  cells  of  the  organizational  interaction  matrix, 
Figure  5.  Of  course,  that  quantification  or  evaluation  may  involve  subjective 
judgement.  The  details  of  the  quantification  process  and  the  numeric  nature 
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of  the  scaling  are  arbitrary  at  this  point  as  long  as  one  can  presume  there  is 
a  monotonically  increasing  relationship  between  factor  values  and  facilitation 
of  technology  transfer  and  user  acceptance.  That  is,  greater  inclination  to 
seek,  hear,  read,  utilize,  etc.,  is  assumed  to  correspond  to  a  greater  degree 
of  organizational  effectiveness  regarding  the  utilization  of  innovations. 

Given  this  monotonic  relationship,  we  define  the  organizational  component 
of  our  model  in  equation  1.  The  organizational  component  "0"  is  defined  to  be 
the  linear  weighted  sum  of  the  organizational  interaction  factors. 


w  =  factor  weights 

a  =  organizational  interaction  factors 
i  =  "player  X"  index,  l<i<5 
j  =  "player  Y"  index,  l<j <5 
k  =  "factor"  index,  l<k<9 


The  organizational  interaction  factors  a...  have  been  discussed.  The 

1 J  K 

weighting  factors  w. . ^  are  included  in  order  to  represent  explicitly  that 
not  all  factors  and  not  all  X-Y  relationships  are  equally  influential. 
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INNOVATION  FEATURES 


In  addition  to  organizational  factors,  user  acceptance  of  a  specific 
innovation  or  device  obviously  will  be  influenced  by  the  perceived  features  of 
the  innovation  or  device  itself.  These  innovation-specific  features  will  be 
subjectively  evaluated  by  users,  and  both  common  sense  and  the  research 
literature  tell  us  that  user  acceptance  is  profoundly  affected  by  this 
evaluation  (Mackie,  et.al.,  1972).  Consequently,  our  model  of  innovation 
acceptance  includes  a  component  which  reflects  specific  features  of 
innovations. 

The  features  we  believe  to  be  particularly  important  in  the  innovation 
acceptance  process  include: 

1.  relative  advantage 

2.  compatabil ity 

3.  simplicity 

4.  observability 

5.  trial  ability 

6.  supportabil ity 

Each  of  these  will  be  briefly  discussed  in  turn.  (Some  are  also  discussed  in 
Section  Two:  Review  of  the  Literature.) 

Relative  Advantage.  Relative  advantage  is  the  degree  to  which  an 
innovation  seems  to  a  user  better  than  the  idea  is  supercedes.  The  greater 
the  relative  advantage,  the  more  rapidly  an  innovation  is  adopted.  This  is 
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not  only  intuitively  clear,  it  has  been  demonstrated  (Rogers  and  Shoemaker, 
1971 ). 


Compatabil ity.  Compatabil ity  is  defined  in  innovation  acceptance  theory 
as  the  degree  to  which  an  innovation  is  seen  by  users  to  be  consistent  with 
their  existing  values,  past  experience,  and  needs.  Compatabil ity  in  the  sense 
that  an  innovation  must  be  operationally  compatible  with  other  systems  with 
which  it  must  work  is  also  of  concern.  The  compatabil  ity  of  an  innovation  has 
been  shown  to  be  positively  related  to  its  rate  of  adoption  (Rogers  and 

Shoemaker,  1971). 

Simp! icity.  Simplicity  is  the  degree  to  which  an  innovation  is  perceived 
as  relatively  easy  to  understand  and  use.  Complexity  has  been  shown  to  be 

negatively  related  to  rate  of  adoption  (Rogers  and  Shoemaker,  1971).  It  has 
been  shown  that  simplicity  is  a  highly  desired  feature  by  users  of  innovative 
systems  in  the  Navy,  although  simplicity  must  be  constrained  by  realistic 
considerations;  if  users  perceive  that  a  problem  has  been  oversimplified  to 
the  point  of  compromising  real-life  complexities,  relative  advantage  is 
sacri ficed. 

Observabil i ty.  Observability  is  the  degree  to  which  the  results  of  an 
innovation  are  visible  to  others.  The  observability  of  an  innovation  has  been 

shown  to  be  positively  related  to  its  rate  of  adoption  (Rogers  and  Shoemaker, 

1971,  p.  168).  (This  of  course  assumes  that  relative  advantage, 
compatabi 1 ity ,  and  simplicity  are  viewed  favorably.)  Observability  is  likely 
to  be  particularly  important  in  acceptance  of  computer  based  products  about 
which  there  is  some  initial  scepticism. 
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Trial  ability.  Trialability  is  the  degree  to  which  an  innovation  may  be 


experimented  with  by  users.  The  trialability  of  an  innovation  has  been  shown 
to  be  positively  related  to  its  rate  of  adoption.  It  should  be  noted  that 
trialability  may  involve  hands-on  experimentation;  however,  it  may  also  be 
vicarious.  That  is,  the  user  may  simple  visualize  how  the  innovation  may  be 
used  in  his  own  operating  environment  by  himself  or  others. 

Supportabil ity .  User  acceptance  is  behavior  which  should  be  sought,  not 
just  at  some  specific  moment  in  time,  e.g.,  at  introduction,  but  over  the 
entire  life  of  the  innovation.  Consequently,  supportabil ity  of  the  innovation 
is  crucial.  Innovations  which  offer  relative  advantage,  compatability, 
simplicity,  etc.  may  still,  ultimately,  experience  user  rejection  if  the 
features  of  the  innovation  cannot  be  maintained  in  a  favorable  condition 
because  of  design  deficiencies,  logistical  support  deficiencies,  lack  of 
timely  updating,  etc. 

FEATURE  COMPONENT  OF  THE  MODEL 


Conceptually,  each  of  the  features  of  innovations  discussed  above  can  be 
quantified.  The  range  and  other  constraints  of  this  quantification  are 
arbitrary  at  this  point  so  long  as  there  is  a  monotonically  increasing 
relationship  between  the  numerical  evaluation  of  a  given  feature  and  likely 
degree  of  user  acceptance. 
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Given  this  assumption,  the  feature  component  of  the  model  is  defined  by 


equation  2: 


b  =  Innovation  feature  values 
c  =  Feature  coefficients 
n  =  Feature  index,  l<n<6 


(2) 


The  feature  component  "F"  is  defined  to  be  the  weighted  linear  combination  of 
the  innovation  feature  values.  The  feature  coefficients  cn  must  be  present 
in  order  to  represent  explicitly  that  the  various  features  of  innovations  are 
not  necessarily  of  equal  importance. 

INNOVATION  ACCEPTANCE  INDEX 

The  predictive  model  of  research  utilization  and  innovation  acceptance  is 
comprised  of  the  organizational  component  and  the  innovation-specific 
component  as  expressed  in  equation  3. 


0  =  organizational  component 


(3) 


F  =  feature-specific  component 
aijk  =  organizational  interaction  factors 
w 

ijk  =  factor  weights 


b 


innovation  feature  values 


n 


c 


n 


feature  coefficients 


K  ,Kf  =  balance  constants 
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The  innovation  acceptance  index  "A"  is  defined  to  be  the  linear  weighted 
sum  of  the  organizational  component  and  the  innovation-specific  component, 
each  of  which  was  previously  described.  The  constants  Kg  and  Kp  are 
balance  constants  to  represent  explicitly  that  the  organizational  component 
and  the  feature-specific  component  do  not  necessarily  have  the  same  impact  on 
research  utilization  and  innovation  acceptance.  For  example,  in  cases  where 
some  features  of  an  innovation  are  viewed  negatively,  it  is  unlikely  that 
organizational  variables  can  insure  acceptance,  no  matter  how  favorable  they 
may  be.  Conversely,  if  the  features  of  an  innovation  have  clear  relative 
advantage  over  what  the  user  currently  employs,  it  is  unlikely  that  poor 

organizational  factors  will  necessarily  prevent  adoption  of  the  innovation. 
Consequently,  the  innovation-specific  component  may  be  weighted  more  heavily 
than  the  organizational  component.  However,  for  many  innovations,  whose 
advantages  may  not  be  immediately  obvious,  the  organizational  factor  is  very 
important  and  can  be  decisive. 

If  the  assumed  monotonic  relationships  between  the  hypothesized  factors 
and  likely  degree  of  acceptance  hold,  ( i . e . ,  if  increasing  any  one  of  them 
increases  likely  degree  of  acceptance),  and  if,  as  assumed,  the  factors 
themselves  are  not  negatively  correlated  to  any  practical  degree  (i.e.,  if 

increasing  one  of  them  does  not  decrease  another  by  a  substantial  amount), 
then  the  numerical  value  of  the  innovation  acceptance  index  "A"  should  be 

strongly  correlated  with  likely  degree  of  user  acceptance. 

Thus,  if  these  assumptions  are  for  practical  purposes  valid,  the  model 

will  predict  likely  degree  of  innovation  acceptance.  The  constraints  on  the 
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constants,  the  range  of  values  employed  in  quantifying  the  variables,  and  any 
subsequent  normalization  procedures  and  additive  constants  can  be  tailored 
arbitrarily.  For  example,  the  innovation  acceptance  index  "A"  can  be  caused 
to  range  between  zero  and  one  hundred,  to  represent  predicted  degrees  of 
acceptance  ranging  from  the  worst  likely  outcome  to  be  experienced,  to  the 
best. 

SUMMARY  AND  CONCLUSION 

A  predictive  model  of  research  utilization  and  innovation  acceptance  has 
been  presented  which  has  an  organizational  component  and  an 
innovation-specific  component.  The  model  takes  account  explicitly  of 
variations  in  relative  impact  of  various  organizational  factors;  player 
pairings;  innovation  features;  and  generic  versus  specific  influence. 
Asymmetrical  relationships  among  players  is  considered  explicitly. 

We  believe  this  model  can  be  useful  in  generating  hypotheses,  structuring 
research,  predicting  the  results  of  change,  and  identifying  areas  where 
research  may  have  the  greatest  impact. 
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SECTION  FIVE 


INTERPRETATION  OF  INTERVIEW  COMMENTS 
IN  THE  CONTEXT  OF  THE  MODEL 

In  the  previous  section  we  described  a  predictive  model  of  innovation 
acceptance  with  particular  reference  to  factors  likely  to  affect  training 
device  utilization.  In  this  section  we  will  attempt  to  interpret  some  of  the 
results  of  our  interviews  with  Navy  personnel  in  the  context  of  the  model  in 
order  to  understand  better  both  the  applicability  of  the  model  and  the  meaning 
of  comments  made  by  the  interviewees.  The  comments  will  be  presented  as 

direct  quotations,  then  interpreted  in  the  light  of  the  model.  These 
quotations  should  not  necessarily  be  considered  to  represent  how  things 
"really  are,"  but  they  certainly  reflect  the  attitudes  and  beliefs  of  some  of 
the  people  interviewed.  Clearly,  these  attitudes  and  beliefs  are  of 

considerable  importance  to  innovation  acceptance. 

In  accordance  with  the  structure  of  the  model,  we  will  consider  comments 

relating  primarily  to  organizational  factors  first:  then,  we  will  consider 

comments  relating  to  features  of  innovations. 

ORGANIZATIONAL  FACTORS 


The  comments  made  to  us  by  the  interviewees,  have  been  organized  roughly 
according  to  the  organizational  categories  identified  in  the  model,  namely 
researcher,  funder,  developer,  builder  and  user.  We  will  discuss  comments 
relating  to  each  of  these  categories  in  turn. 
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Researcher 


"Many  of  the  researchers  in  this  laboratory  don't  know  where  their 
research  is  going.  The  most  successful  people  here  have  been  the  ones  who 
have  been  willing  to  get  out  and  work  with  other  agencies  and  offices." 

Obviously,  the  researcher  who  made  this  comment  believes  that  a  high 
degree  of  organizational  interaction  is  desirable,  and  that  such  interaction 
is  correlated  with  "success." 

"The  transfer  of  R&D  to  device  design  is  not  very  good.  [Researchers] 
could  have  an  impact  but  they  need  to  get  over  their  isolationism." 

Another  believer  in  the  desirability  of  enhanced  organizational 
interaction,  this  member  of  the  funder  community  believed  that  the  problem  is 
more  the  result  of  researchers'  reluctance  to  interact  rather  than  reluctance 
on  the  part  of  other  players  (funder,  developer,  builder,  user). 
Quantification  of  the  organizational  interaction  matrix,  Figure  5,  would 
provide  the  opportunity  to  see  whether  other  players  view  researchers  in  this 
way  also. 

"There  are  difficulties  in  translating  research  results  into  formats  that 
the  engineers  are  used  to.  R&D  people  should  be  more  involved  in  the 
translating  process." 

In  the  context  of  our  model,  this  comment  by  a  researcher  refers  to  the 
researcher-developer  and  the  researcher-builder  cells  of  the  organizational 
interaction  matrix  on  factors  6  and  7,  "talk  innovation  information,"  and 
"write  innovation  information." 

"There  appear  to  be  many  innovative  devices  and  new  developments  sitting 
around  that  we  don't  get  to  use.  Where  is  the  clearinghouse  for  these 
i nnovations?" 
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As  we  mentioned  in  the  discussion  of  the  generalized  organizational 
diagram,  Figure  3,  the  research  function  in  the  Navy  is  performed  within  a 
variety  of  organizations  in  a  number  of  different  places.  There  would  indeed 
appear  to  be  a  problem  in  keeping  abreast  of  developments  across  the  board,  as 
this  member  of  the  user  community  commented. 

"The  design  [of  a  certain  trainer  constructed  for  research  purposes] 
reflected  a  lack  of  awareness  of  how  training  is  being  done.  Strong 
rejection  followed.  You  need  a  hotline  among  all  interested  parties." 

Another  strong  endorsement  of  organizational  interaction  and  an  indication 
of  the  possible  consequences  of  its  absence. 

"We're  solving  problems  on  line  and  are  way  ahead  of  the  researchers." 

This  comment  by  an  engineer  of  the  "developer"  community  probably  reflects 
a  different  criterion  of  evaluation  than  the  researchers  themselves  would 
employ.  However,  the  important  thing  is  the  attitude  that  researchers  are 
lagging  behind  in  terms  of  something  that  the  engineer  judges  to  be  relevant 
to  his  needs.  Although  there  is  considerable  ground  for  philosophical 
differences  concerning  what  the  objectives  of  R&D  should  be,  frequently 
attitudes  such  as  the  one  reflected  in  this  quotation  can  result  simply  from 
poor  communications  between  the  researcher  and  the  developer. 

"Most  R&D  shows  up  in  proposals  in  such  a  way  as  to  generate  a  competetive 
edge.  This  may  be  the  fastest  way  to  generate  an  impact  from  research." 

Thus,  it  may  be  the  "builder"  that  facilitates  the  introduction  of 
innovation.  This  is  likely  the  result  primarily  of  contractor  in-house  R&D, 
and  raises  interesting  questions  regarding  the  relationship  between  contractor 
R&D  and  Navy  research  efforts.  In  our  model,  the  upper  left  hand  cell  of  the 
organizational  interaction  matrix.  Figure  5,  (researcher-researcher)  is  meant 
to  represent  interactions  within  the  research  community. 
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Navy  R&D  people  should  be  doing  more  demonstrations  and  fewer  reports. 


This  comment  by  a  researcher  stems  from  experiences  indicating  that 
informal  communications  and  demonstrations  can  be  more  effective  than  written 
communications.  Whether  or  not  R&D  people  should  write  fewer  reports,  it  does 
seem  likely  that  researchers  might  be  able  to  communicate  more  effectively 
with  the  other  players  (funder,  developer,  builder,  user)  via  other  channels. 
The  model  is  intended  to  recognize  this  explicitly. 

"When  the  research  people  are  called  in  for  a  problem  it  is  too  late  to  do 
much  good." 

Acquisition  programs  are  almost  always  conducted  on  an  extremely  stringent 
time  schedule.  However,  the  scientific  approach  to  problem  solving  often 
involves  time-consuming  empirical  research  if  clearly  applicable  data  do  not 
already  exist.  If  researchers  and  developers  had  a  more  highly  interactive 
relationsip,  perhaps  problems  could  be  anticipated  more  often,  so  that  needed 
data  could  be  generated  in  a  timely  fashion. 

"One  thing  that  limits  the  use  of  innovations  from  research  is  the  need 
for  demonstration  that  the  data  apply  in  the  new  context". 

Sometimes,  this  problem  arises  because  experimental  tasks  are  chosen  which 
are  not  relevant  to  any  specific  application.  Often,  the  basic  scientific 
objectives  can  be  addressed  employing  experimental  tasks  that  are 
operationally  relevant.  Improved  organizational  interaction  may  be  a 
sufficient  condition  for  improving  relevance;  it  surely  is  a  necessary 
condition. 

"The  real  means  of  relating  research  to  real  world  problems  is  through  a 
continuing  dialogue." 
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As  discussed  in  Section  4,  the  model  is  intended  to  permit  representation 
of  a  variety  of  organizational  factors  and  players  involved  in  achieving  an 
effective  dialogue. 


"There  are  fundamental  differences  and  viewpoints  between  users  and 
researchers  concerning  trainer  features  necessary  to  achieve  particular 
training  objectives." 


We  hypothesize  that  a  significant  proportion  of  the  differences  referred 
to  by  this  researcher  could  be  alleviated  by  improved  organizational 
l nteraction. 


Funder 


"There  should  be  budgeting  provisions  for  insuring  product  improvement  and 
necessary  changes  after  a  device  is  first  delivered.  This  should  be  a 
cradle-to-grave  support  program." 

We  noted  frequent  expressions  like  this  one  from  a  member  of  the  developer 
community  addressing  the  need  for  adequate  support  of  innovations  after  they 
are  in  the  field.  Largely,  this  support  is  dependent  on  the  sponsorship  of 
the  funder,  and  is  forthcoming  only  if  the  funder  is  willing  and  able  to 
"compete"  successfully  for  funds  against  newer,  more  urgent— or  more 
"gl  amorous"— systems. 

"There  are  a  whole  bunch  of  formal  directives  that  nobody  lives  with." 

Formal  directives,  in  themselves,  do  not  get  things  done.  Command 
attention  and  informal  interaction  play  very  important  roles  in  the  way  things 
are  really  done.  However,  formal  directives  should  support  and  reflect  actual 
practice,  or  they  can  be  counter-productive.  If,  as  this  developer  stated, 
they  don't  reflect  actual  practice,  the  situation  needs  close  examination  by 
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management. 


"[The  developer]  appears  to  be  caught  up  in  a  time  schedule.  They  won't 
tolerate  any  delay  even  if  research  needs  to  be  done." 

Time  pressure,  indeed,  appears  to  be  very  great,  particularly  with 
training  systems  that  must  be  in  place  to  support  new  weapon  systems. 
Hov/ever,  the  "funder,"  as  the  executive  and  overall  program  planner  in  our 

"researcher,  funder,  developer,  builder,  user"  model,  needs  to  employ  every 
management  device  possible  to  maximize  the  availability  of  resources  (time, 
money)  to  ensure  the  systematic  and  orderly  development  of  training  systems. 
The  quotation  above  by  a  "user"  indicates  that  he  does  not  feel  that  an 
optimum  balance  has  been  achieved. 

"It  is  difficult  to  get  the  detailers  to  recognize  the  importance  of 
leaving  Fleet  Project  Team  members  in  their  assignments." 

Lack  of  continuity  in  Fleet  Project  Team  membership  was  one  of  the  most 
frequent  comments  we  received.  Management  (i.e.,  OPNAV)  should  take  a  close 
look  at  this  problem,  or  if  they  have,  they  should  attempt  to  communicate 

convincingly  to  the  Fleet  that  the  present  system  is  optimal.  We  suspect 
that,  in  the  context  of  our  model,  communication  between  "funder"  and  "user" 
on  this  issue  leaves  something  to  be  desired.  The  FPT  member  who  made  the 
comment  above  certainly  felt  that  way. 

"The  responsi bi 1 ity/authori ty  for  procurement  of  [a  large  combat  systems 
trainer]  as  a  whole  is  far  too  diffused.  Nobody  is  looking  at  the  fully 
i ntegrated  system." 

We  heard  a  number  of  comments  like  this  one  from  a  member  of  the  user 

community  to  the  effect  that  responsi bil ity/authori ty  for  procurement  for  a 
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variety  of  training  systems  was  too  diffused.  Management  control  is  one  sort 
of  communication,  a  kind  of  feedback  loop  that  shapes  the  behavior  of  various 
elements  of  an  organization  to  bring  about  coherent  movement  toward  overall 
goals.  There  is  the  impression  among  a  number  of  people  we  talked  to  that 

this  sort  of  management  control  is  insufficient  during  the  development  of  many 
training  devices. 

"The  required  computer  components  were  not  in  the  supply  system.  Lack  of 
full  support  has  resulted  in  degrading  the  performance  of  a  very  good 

training  device.  This  reflects  a  management  failure;  it's  a  good  example 
of  responsibility  ending  when  the  device  is  delivered." 

The  sort  of  problem  pointed  to  by  this  user  comment  obviously  needs 
attention.  Even  if  everything  possible  is  being  done,  the  perception 
in  the  field  is  sometimes  that  nothing  is  being  done.  This  fosters  an 
attitude  which  certainly  does  not  enhance  acceptance. 

"QA&R  is  never  called  in  to  the  early  device  acquisition  process. 

We  caution  that  they  shouldn't  reinvent  the  wheel,  but  have  never 
been  invited  in." 

An  example  of  where  "X"  is  willing  to  talk,  but  "Y"  is  unwilling  or  unable 

to  hear?  Or  has  "Y"  been  able  to  obtain  the  information  somewhere  else?  If 

so,  "X"  is  uninformed  about  it,  and  believes  there  has  been  a  communications 
breakdown. 

"We  must  deliver  training  systems,  not  devices,  if  we  are  to  avoid  user 
acceptance  problems." 

This,  of  course,  is  a  very  important  point  (made  by  a  member  of  the 
developer  community),  and  the  Navy  management  should  make  a  concerted  effort 
to  assure  that  training  systems  are  indeed  delivered.  In  the  context  of  the 
model,  it  will  require  that  the  delivery  of  training  systems  be  facilitated  by 
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making  the  appropriate  information  and  other  resources  available,  and  by 
making  it  rewardi ng  for  the  developer  to  do  so. 

"Often  we  don’t  have  adequate  numbers  of  engineers  to  effectively  manage 
the  contractors." 

In  the  context  of  the  model,  a  resource  shortage  of  the  sort 
pointed  out  by  this  member  of  the  "developer's"  organization  negatively 
impacts  the  ability  of  the  "developer"  to  interact  with  the  "builder," 
with  resulting  potential  for  negative  impact  on  eventual  user 
acceptance. 

"Some  trainers  meet  the  specifications  very  well  but  are  poor 
trainers  because  the  specs  were  bad.  Specifications  are  often 
vague  in  many  areas." 

This  comment  by  a  member  of  the  "user"  community  was  echoed  by  others  we 
interviewed.  We  believe  the  most  likely  cause  for  this  problem  is  inadequate 
front-end  analysis  and  insufficiently  supported  functional  specification.  The 
probable  cause  of  these  shortcomings  is  the  time  pressure  which  is  pervasive 
throughout  the  acquisition  cycle  of  trainers  which  are  intended  to  support  new 
weapon  systems.  Compromises  made  in  the  early  stages  of  the  acquisition  cycle 
are  particularly  likely  to  have  substantial  negative  consequences  in  eventual 
training  effectiveness  and  user  acceptance. 

"There  should  be  an  on-line  reporting  system  that  permits  users  to  get 
directly  back  to  the  procurement  system." 

This  comment  by  a  "funder"  suggests  that  an  interactional  asymmetry 
exists  in  the  developer-user  relationship.  However,  improved  opportunities 
for  user  feedback  to  the  developer  are  not  likely  unless  the  added  effort 
required  to  provide  them  is  perceived  by  the  developer  to  be  sufficiently 
rewarding.  Often,  the  immediate  payoff  to  the  developer  is  in  solving 
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problems  with  current  acquisition  projects,  rather  than  projects  which  are 
perceived  to  be  "completed."  Positive  reinforcement  of  "developer's" 
increased  attention  to  "user"  after  a  trainer  has  been  delivered  is  likely  to 
be  provided  effectively  only  by  top  management— the  "funder."  Otherwise,  it 
is  easy  to  understand  that  the  developer  will  be  preoccupied  with  problems  of 
the  system  he  is  currently  developing,  while  the  user  is  preoccupied  with 
problems  of  the  already-developed  system  which  he  has  to  employ. 

"[This  developer]  needs  to  talk  more  within  itself  about  the  state  of  the 
art  and  what  we're  doing." 

Our  model  provides  for  representation  of  interactions  wi thi n 
organizational  elements,  along  the  diagonal  of  the  organizational  interaction 
matrix,  Figure  5.  For  example,  this  interviewee  was  commenting  on 
interactions  that  would  be  represented  in  the  "developer-developer"  cell. 

"System  sponsors  say  'get  the  hardware  out  to  the  Fleet  and  they  will 
figure  out  how  to  use  it.'" 

Although  the  Navy's  traditional  "can  do"  attitude  does  sometimes  foster 
this  sort  of  thinking,  it  is,  of  course,  not  consistent  with  an  effective 
approach  to  the  introduction  of  training  innovations  and  training  systems 
implementation.  Unfortunately,  we  believe  this  comment,  made  by  member  of  the 
"developer"  community,  all  too  often  represents  actual  practice  quite 
accurately. 

"It  is  practically  never  possible  to  do  the  required  front-end  analysis. 
It  has  to  be  done  some  five  years  before  the  training 
device  will  exist  to  meet  budgetary  cycle  constraints.  The  contract 
should  be  let  four  years  before  the  aircraft  flies." 

This  is  an  extremely  difficult  problem  to  solve,  as  this  "developer's" 
comment  suggests,  but  clearly  it  can  have  a  negative  impact  on  eventual  user 
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acceptance.  Perhaps  one  thing  that  can  be  done  to  ameliorate  the  acceptance 
situation  is  to  make  clear  to  the  user  community  what  the  problem  is  and  how 
difficult  it  is  to  solve. 

"We  have  hundreds  of  millions  of  dollars  of  trainers  in  the  field  that 

were  requested  by  users  who  were  gone  by  the  time  they  were  delivered. 

The  device  ends  up  with  no  [user]  support." 

Military  personnel  turnover  is  a  fact  of  life,  as  this  developer's  comment 
indicates.  However,  the  negative  consequences  of  this  turnover  can  be  reduced 
to  a  minimum  if  the  training  system  funder  and  developer  have  a  continuing 
commitment  to  fielded  systems,  and  ensure  that  a  dialogue  is  established  with 
new  users,  so  they  can  at  least  appreciate,  if  not  totally  endorse,  the 
reason-for-being  of  trainers  they  "inherit." 


Developer 

"[This  SYSCOM]  needs  to  get  more  involved  in  how  programs  in  6.1,  6.2  and 
6.3  [i.e.,  research  programs]  might  help  them  downstream". 

This  comment  reflect  a  felt  need  on  the  part  of  one  member  of  a 
"developer"  organization  for  closer  interaction  with  researchers. 

"A  successful  project  manager  will  have  a  strong  communication  link  with 
the  chairman  of  the  Fleet  Project  Team." 

Obviously,  this  developer  believes  in  the  importance  of  interaction  with 
the  user. 


"FASO  [Fleet  Aviation  Specialized  Operational  Training  Group]  has  the 
responsibility  to  track  configuration  changes  and  let  the  NTEC  [Naval 
Training  Equipment  Center]  Project  Engineer  know  what  changes  need  to  be 
made.  There  does  not  not  seem  to  be  a  solid  method  for  ensuring  that  this 
will  happen.  We  have  a  4  to  5-year  accumulation  of  problems  identified  by 
QA&R  [Quality  Assurance  and  Revalidation]  that  have  not  been  fixed". 
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It  is  evident  that  this  sort  of  perceived  situation  is  likely  to  have  a 
negative  impact  on  user  attitudes;  it  certainly  had  on  the  user  who  made  this 
comment.  If  there  is  a  good  reason  for  the  developer's  inability  to  support 
trainers  in  the  field,  those  reasons  should  be  communicated  effectively  to  the 
users  to  head  off  the  formation  of  negative  attitudes. 


"[This  SYSCOM]  does  not  have  an  effective  tie  with  what  the  government 
labs  are  doing.  It  is  doubtful  whether  we  influence  what  goes  on  in 
research  codes." 


In  the  context  of  our  model,  this  opinion  reflects  a  deficient 
researcher-developer  i nteraction. 


"MC's  [Military  Characteristics--specifications  from  which  trainer 
procurement  contracts  are  prepared]  are  kind  of  ginned  up  to  meet  the 
Fleet's  wish  list.  There  is  not  enough  front-end  analysis.  There  does 
not  seem  to  be  a  clear  cut  criterion  for  how  much  front  end  analysis  will 
be  done.  [The  developer]  often  skips  the  process  altogether  and  jumps 
right  into  development." 


We  don't  know  how  accurate  this  comment  by  a  training  technology 
researcher  is,  but  from  our  interviews  it  does  seem  that  time  pressure  makes 
it  expedient  to  move  into  development  as  quickly  as  possible,  and  the 
atomosphere  certainly  lends  itself  to  glossing  over  systematic  front-end 
analyses.  However,  to  the  extent  that  such  compromises  of  systematic 
development  are  made,  greater  risk  is  taken  regarding  actual  training 
effectiveness  and  eventual  user  acceptance. 


"[The  developer's]  engineers  evaluate  their  own  job!  Acceptance  testing 
should  be  done  by  someone  else." 

Actually,  the  developer's  engineers  may  be  the  best  person  to  judge 
whether  the  product  has  met  the  specification.  However,  as  mentioned 
elsewhere,  the  specification  may  not  necessarily  be  an  accurate  translation  of 
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what  the  training  objectives  are.  Therefore,  it  would  seem  desirable  that  in 
addition  to  acceptance  testing  against  the  contractual  specification,  there 
should  be  some  sort  of  "operational  test  and  evaluation"  and/or  training 
capabilities  testing  performed.  This  would  probably  not  only  lead  to  a  better 
product,  but  serve  as  well  to  allay  the  doubts  of  people  in  the  user  community 
like  the  one  who  made  the  comments  above.  CNET's  QA&R  program  is  doing  a 
review  of  devices  just  after  contractual  acceptance  in  conjunction  with  the 
Fleet  Project  Team.  This  may  serve  to  answer  the  need,  but  the  practice  is 
not  yet  firmly  established  Navy-wide. 

"The  front  end  analyst  must  put  in  features  that  he  knows  are  desirable 

and  then  sell  them  to  the  user." 

We  are  sympathetic  with  this  point  of  view  expressed  by  a  member  of  the 
"developer's"  community;  if  the  training  specialist  has  done  his  homework  and 
has  interacted  closely  with  the  user,  it  is  likely  he  will  be  able  to  identify 
features  that  are  desirable  in  terms  of  meeting  training  objectives.  However, 
when  innovations  are  involved,  he  still  may  have  to  "sell"  them  to  the  user. 
This  is  a  role  that  is  not  comfortable  for  many  developer  personnel.  In  any 
case  the  whole  process  implies  a  high  degree  of  organizational  interaction 
between  the  developer  and  the  user. 

"Training  course  documentation  is  always  the  weakest  part  of  the  package 

when  it  is  delivered." 

"Documentation  in  how  to  use  a  trainer  is  never  up  to  snuff." 

"There  is  no  instruction  on  how  to  instruct  using  training  devices." 

These  comments  by  members  of  the  developer  community  have  been  echoed  by 
others.  The  organizational  reward  system  is  much  more  likely  to  focus  on  the 
hardware  parts  of  the  training  product  in  reinforcing  developer  behavior. 
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However,  particularly  with  the  ever-increasing  sophistication  of  trainers  ,  it 
is  not  reasonable  to  expect  the  user  to  figure  out  for  himself  how  best  to  use 
a  trainer.  Members  of  the  developer  community  recognize  this  (as  the  above 
quotations  demonstrate),  but  they  cannot  change  the  organizational  system  of 
reward  and  punishment  by  themselves;  it  will  take  interaction  among  the 
various  players,  and  particularly  the  support  of  top  management,  (i.e.,  the 
funder) . 


We  received  many  comments  regarding  methods  of  training  system 
development.  Several  pertinent  quotations  are  presented  below,  followed  by 
our  own  commentary. 


"The  trainer  contract  may  call  for  a  front-end  analysis-- this  is  too  late."' 

"DOD  instructions  now  exist  that  require  front-end  analysis.  This  doesn't 
ensure  that  it  will  always  be  done  because  of  time  and  budget  constraints, 
but  usually  it  is." 

"You  often  have  to  fight  for  the  front-end  analysis.  The  educational 
specialists  and  the  engineer  really  look  for  different  things,  and  both 
are  essential  as  a  complimentary  team." 

"[The  developer]  should  do  much  more  front-end  analyses  of 

objectives/needed  media  types." 

"Front-end  analysis  is  being  done  after  the  fact." 

"[This  developer]  sees  a  need  for  front-end  analysis  but  feels  ISD 

[Instructional  Systems  Development]  has  become  a  rigid  step-by-step 
process  that  costs  a  lot  of  dollars  to  produce  trivial  outputs.  ISD 

should  focus  on  the  areas  of  innovation." 

"The  front-end  analysts  are  not  of  uniform  quality.  The  engineers  are 
usually  quite  good  but  do  not  know  how  to  do  the  front-end  analysis  or 
supervise  contractors  who  perform  it." 

"We  are  being  selective  in  the  amount  of  ISD  but  usually  carry  it  from 

task  analysis  through  media  selection." 

"Trainers  are  developed  prior  to  identifying  tasks  that  have  to  be 
taught.  The  training  objectives  are  very  general." 
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"We  have  always  had  OFT's  (historically);  therefore,  it  is  very  hard  to 
reorient  user  thinking  toward  part-task  trainers.  ISD  has  made  a 
significant  change  in  this  respect  but  it  is  uncertain  as  to  how  long  the 
ISD  impact  will  last." 


These  comments  by  researchers,  developers,  and  users  indicate  a  problem 
area.  Based  on  the  interviews,  we  hypothesize  that  the  application  of 
potentially  effective  methods  of  training  system  development  (e.g.,  front-end 
analysis,  selected  aspects  of  instructional  systems  development  (ISD))  is 
beset  by  three  principal  difficulties: 


1.  Difficulty  of  gaining  recognition  that  the  process  requires  specialized 
knowledge.  Many  of  the  most  promising  methods  of  training  system 
design  have  grown  out  of  basic  research  in  management  and  behavioral 
sciences.  A  substantial  body  of  specialized  knowledge  has  been 
developed  in  areas  which  often  appear  to  the  non-specialist  as 
requiring  nothing  more  than  "common  sense"  or  "intuition"  to  deal  with 
effectively.  Thus,  "hardware-oriented"  managers  often  do  not 
appreciate  the  contribution  that  is  potentially  available.  Certainly, 
common  sense  is  required  in  large  doses.  But  intuition  has  been  shown 
very  often  to  be  a  poor  substitute  for  specialized  knowledge  of  human 
behavior  when  attempting  to  build  training  systems  that  will  shape 
human  behavior  effectively. 

2.  Bad  impressions  created  by  some  training  specialists.  Some  training 
specialists  have  tried  to  apply  methods  of  instructional  systems 
development  without  really  coming  to  understand  the  subject  matter  area 
of  the  training  system,  and  the  real  world  constraints  on  it, 
sufficiently.  A  slavish  devotion  to  the  principles  of  ISD  without  a 
genuine  attempt  to  understand  either  the  subject  matter  area  or  the 
problems  of  the  training  systems  manager  and  user  is  virtually  certain 
to  lead  to  an  inappropriate  product  and  to  alienation  of  the  other 
pi  ayers . 

3.  Ever-present  time/money  pressure. 


We  believe  a  systems  approach  to  training  development  is  essential.  However, 
the  systems  approach  is,  in  some  part,  innovative,  and  the  acceptance  and 
endorsement  of  these  innovations  by  the  "funder"  and  "developer"  is  essential 
for  their  continued  effective  use.  This  means  that  educational  specialists 
and  others  who  wish  to  promote  development  of  innovative  training  systems  must 
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engage  in  an  optimum  interaction  with  the  funder  and  the  "traditional"  or 


"hardware"  side  of  the  developer's  house. 


Builder 


"In  some  cases  the  contractor  is  a  leader,  if  he  really  knows  his  stuff; 
in  others  he  is  a  follower  -  responds  to  everybody's  inputs." 

"Lots  of  civilian  contractors  build  trainers  that  won't  do  many  of  the 
things  they  should  do  because  they  really  don't  understand  how  the 
[operational]  system  works." 

"The  contractor  often  has  no  knowledge  about  the  subject  matter  of  the 
trainer.  He  often  doesn't  understand  critical  details  [in  stimulus 
presentation] . " 

It  was  frequently  commented  to  us,  as  in  these  remarks  from  the  developer 
and  user  communities,  that  contractors  were  the  source  of  considerable 
variability  in  the  quality  of  training  systems.  It  appears  that  the 
procurement  process  cannot  guarantee  selection  of  a  contractor  who  will 
produce  a  satisfactory  product.  Sometimes,  the  competetive  process  is  such 

that  the  developer  must  accept  a  contractor  as  "winner"  of  the  competition 

even  though  he  doubts  the  contractor  can  totally  live  up  to  the  claims  of  his 
proposal.  In  other  instances,  the  same  contractor  who  has  performed  well  on 
past  projects  may  perform  poorly;  or  vice  versa.  It  appears  to  us  that 

although  some  variability  is  unavoidable,  a  strong  interaction  between 

"developer"  and  "builder"  will  go  far  to  make  the  best  of  the  situation.  The 
above  quotes  also  suggest  that  a  strong  interaction  between  "builder"  and 
"user"  is  desirable. 


"The  source  of  most  innovative  technology  would  appear  to  be  the 
contractor's  response  to  the  military  characteristics/device 
specifications. " 


82 


It  is  interesting  that  we  frequently  received  comments  like  this  one  from 
a  "developer"  to  the  effect  that  training  device  innovation  is  more  likely  to 
arise  from  the  private  sector  than  from  the  military  laboratory  or  research 
agencies.  It  is  understandable,  and  in  fact  a  benefit  of  the  free  enterprise 
system,  that  innovation  is  brought  forth  by  the  contractor  to  gain  a 
competitive  edge.  But  the  perceived  leadership  of  the  private  sector  in 
innovation  causes  some  members  of  the  developer  community  to  question  the 
value  of  governmental  research  activities. 

We  did  not  have  the  opportunity  to  interview  any  persons  from  the  private 
sector  "builder"  community.  Consequently,  we  cannot  offer  any  insight 
directly  from  the  "builder's"  point  of  view.  It  would  be  desirable  to  expand 
the  data  base  to  include  those  points  of  view. 

User 


"User  non-acceptance  almost  always  reflects  a  lack  of  proper  introduction 
of  the  device  to  the  user." 


Most  researchers  who  have  studied  innovation  acceptance  would  agree  with 
the  researcher  who  made  this  comment  that  proper  introduction  of  innovations 
is  a  necessary  condition  for  acceptance  in  most  cases. 

"The  FPT  [Fleet  Project  Team]  is  the  key  element  in  the  'ownership' 
issue.  The  qualifications  of  FPT  members  represent  an  uncertainty,  and 
there  is  no  clear  mechanism  for  the  FPT  members'  peers  to  know  what  is 
being  done  on  their  behalf." 

This  perceptive  member  of  the  "developer's"  community  has  pointed  to  one 
of  the  FPT's  important  roles.  The  Fleet  Project  Team  members  act  as 
"linkers,"  which  are  defined  by  Jolly  (1S80)  as  "individuals  in  the  receiving 
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organization  who  link  or  couple  persons  in  their  organizations  to  outside 
ideas,  concepts,  and  new  devices.  Linkers  are  persons  who  operate  in  the  same 
organization  or  allied  organizations  (with  social  overlap)  as  those  persons 
who  will  actually  use  the  new  technology.  Linkers  are  sometimes  called  gate 
keepers,  opinion  leaders,  information  sources,  or  early  knowers  of  knowledge. 
Linkers  are  not  necessarily  superior  technical  persons,  rather  they  are 
"knowledgeable  sources."  A  necessary  part  of  the  linker  role  is  effective 
communication  with  other  persons  in  the  user  organization.  Consequently,  the 
comment  above  regarding  "no  clear  mechanism  for  the  FPT  members'  peers  to  know 
what  is  being  done,"  which  we  believe  to  be  generally  true,  is  particularly 
distressing.  This  weakness  would  be  represented  in  our  model  in  the 
"user-user"  cell  of  the  organizational  interaction  matrix,  Figure  5,  by  poor 
internal  interactions  as  represented  by  the  various  factors. 

"The  sense  of  ownership  is  important  in  relation  to  user  acceptance;  who 

developed  the  idea  is  important". 

This  comment  by  a  member  of  the  developer  community  regarding  user 
acceptance  shows  a  salutary  awareness  of  fundamental  principles  of  innovation 
acceptance.  Whether  these  principles  are  able  to  be  put  into  practice 
effectively  depends  on  positive  organizational  interaction  across  the  board. 

"The  users  must  see  immediate  benefits  to  themselves." 

Immediate  benefit  is  crucial,  as  this  "funder"  has  pointed  out,  and  it 

> 

involves  both  the  organizational  milieu  and  specific  features  of  the 
innovation.  Fleet  personnel  whom  we  interviewed  during  this  project  were  for 
the  most  part  highly  dedicated  to  meeting  their  operational  commitments,  which 
often  impose  substantial  workloads.  Most  fleet  users  have  little  time  or 
inclination  to  adopt  innovations  that  do  not  seem  to  contribute  directly  and 
immediately  to  meeting  these  commitments. 


84 


"User  resistance  is  high  if  the  trainer  was  1)  'not  invented  here  and'  2) 
'does  not  train  in  the  way  I  was  trained  in  the  past.'  User  personnel  are 
not  typically  qualified  to  judge  the  potential  training  effectiveness  of 
the  trainer." 

The  "not  invented  here"  syndrome  is  recognized  by  this  "developer,"  and  is 
well  known  to  those  who  study  innovation  acceptance.  The  persistence  of  old, 
familiar  methods  is  a  backbone  of  resistance  to  innovation.  Regarding  the 
qualifications  of  users  to  judge  potential  training  effectiveness,  many  users 
suspect  that  some  members  of  the  devel oper  community  are  not  qualified  to 
judge  the  potential  training  effectiveness  of  various  innovations  either.  The 
developer  must  attain  credibility  in  the  eyes  of  the  user  to  be  able  to 
promote  innovations  in  the  most  favorable  way.  This  factor  is  represented  in 
the  model . 

"We  [the  fleet]  need  to  make  inputs  on  threats  and  required  fidelity. 
CNET  should  then  take  charge  without  further  inputs  uni  ess  there  are 
trade-off  decisions  about  available  dollars.  The  fleet  should  be  involved 
in  prioritizing;  we  don't  presently  do  this." 

Developer  and  user  agree  that  subject  matter  expertise  is  a  necessary 
input— al though  the  issue  of  who  is  able  to  judge  "required"  fidelity  is  a 
delicate  one.  Regarding  prioritizing,  we  frequently  got  comments  like  this 
from  fleet  users  that  it  was  desireable  and  appropriate  for  them  to  make  such 
inputs.  In  some  cases,  they  have  the  opportunity  to  do  so;  in  others,  they 
feel  their  wishes  have  been  ignored. 

"We  train  primarily  at  sea  because  we  do  not  have  suitable  trainers." 

This  comment  by  a  fleet  training  officer  implicitly  suggests  that 
"suitable  trainers"  would  be  used  to  replace  at-sea  training  hours.  We 
believe  trainers  would  be  used  in  this  officer's  warfare  area  because 
post-exercise  reconstruction  for  the  purpose  of  training  feedback  is  extremely 
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time-consuming  and  costly  to  obtain  from  at-sea  exercises.  Acceptance  is 
greatly  facilitated  when  there  is  a  strong  felt  need.  The  definition  of 
"suitable"  trainers  requires  interaction  between  developer  and  user.  Trainers 
are  being  developed  in  this  area,  but  they  will  not  be  delivered  soon— a  fact 
that  has  generated  considerable  user  criticism. 

"There  is  a  big  difference  between  what  the  school  needs  and  what  they  say 
they  want.  School  personnel  are  not  knowledgeable  about  what's  available." 

As  this  member  of  "developer"  organization  points  out,  fleet  personnel  are 
not  generally  training  technology  experts.  However,  since  school  personnel 
are  performing  the  day-to-day  tasks  of  training  in  the  Navy,  is  to  be  expected 
that  they  will  form  very  definite  opinions  regarding  their  needs.  In  the 
absence  of  a  strong  interaction  between  developer  and  user,  the  user  may 
"invent"  a  different  approach  to  a  problem  than  the  developer,  which  is  bound 
to  lead  to  substantial  acceptance  problems. 


"The  Surface  Warfare  Trainer  Group  is  kind  of  replacing  the  Fleet  Project 
Team  during  the  planning  phase,  with  the  FPT  playing  the  more  important 
role  during  test  and  evaluation.  The  SWTG  mostly  involves  tweeking  of 
already  laid  plans.  However,  their  evaluative  inputs  are  very  valuable, 
and  their  sense  of  involvement  is  very  important." 


We  hypothesize  that  the  Surface  Warfare  Trainer  Group,  the  Submarine 
Trainer/Training  Working  Group,  and  the  Fleet  Project  Teams  are  extremely 
important  in  establishing  organizational  interactions  in  the  Navy  which  are 
favorable  to  innovation  acceptance. 


"The  computer  graphics  in  the  amusement  arcades  is  often  better  than 
ours.  Our  systems  are  antiquated.  You  know  the  technology  is  out 
there--but  we  won't  see  it  for  years." 


The  rapidly  advancing  technology  of  the  electronics  industry  has  spawned 
so  many  new  developments  in  consumer-oriented  electronics  at  such  a  rapid 
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pace,  that  users  of  military  technology  (like  the  one  who  made  the  comment 
above)  find  it  especially  difficult  to  accept  the  lengthy  training  systems 
acquisition  cycle.  There  are  no  doubt  many  good  reasons  why  that  cycle  is 
lengthy— and  perhaps  some  not-so-good  reasons.  Greater  interaction  between 
funders,  developers,  and  users  could  help  the  users  to  be  more  accepting  of 
the  time  necessary  to  develop  new  military  systems.  This  could  be  useful  in 
facilitating  acceptance  of  currently  available  trainers  while  new  ones  are 
being  developed. 

"FPT  members  don't  know  enough  about  the  new  equipment  or  the  theory  of 
operations  to  state  what  the  trainer  should  really  do  [in  the  case  of 
prime  systems  in  the  very  early  stages  of  development].  They  ought  to  go 
to  the  lab  which  is  developing  the  prime  hardware  system." 

This  is  a  request  by  a  user  for  greater  interaction  with  the  developers  of 
new  weapon  systems  for  which  trainers  must  be  built.  It  is  another  example  of 
an  attitude  which  we  encountered  fairly  often,  namely,  the  desire  of  Fleet 
Project  Team  members  to  have  a  stronger  interaction  with  the  other  players. 

"We  sometimes  get  invited  but  can't  go  to  the  meetings  for  the  lack  of 
dollars  or  time— or  else  we  can't  send  the  'right'  person." 

"I  could  easily  spend  90%  of  my  time  meeting  FPT  responsibilities." 

"Sometimes  only  the  east  coast  gets  the  FPT  members  to  a  meeting  because 
their  travel  costs  are  usually  less  [since  they  are  closer  to  NTEC.]" 

"Travel  dollar  problems  are  commonplace." 

"We  have  about  two  weeks  to  review  and  prepare  comments  on  very  complex 
technical  documentation.  The  requirements  can  come  at  very  inopportune 
times. " 

"There  is  never  enough  time  for  an  FPT  member  to  do  the  job  right." 

"As  Chairman  of  an  FPT,  I  am  on  the  phone  three  hours  a  day.  Some 
commands  cannot  afford  this  kind  of  time — it  just  comes  out  of  our  hide." 

"I  have  one  officer  on  the  road  two  weeks  out  of  four  and  he  still  cannot 
do  all  of  the  things  that  the  FPT  Manual  requires." 
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"I  can't  possibly  evaluate  all  the  specs  they  send  me,  especially  in  the 
10-day  period  I  usually  have." 

"It  is  very  difficult  for  fleet  personnel  to  make  the  inputs  they  should 
to  new  training  devices  because  of  lack  of  time  and  dollars." 

These  comments  by  FPT  members  (users)  speak  for  themselves.  Fleet  Project 
Team  membership  is  always  a  collateral  duty,  and  OPNAV  Instruction  1551.7b 
states  that  "travel  and  per  diem  funds  required  for  FPT  members  shall  be 
provided  by  the  members'  respective  duty  stations."  Conscientious  Fleet 
Project  Team  members  frequently  complained  of  the  constraints  of  time  and 
funds . 

"There  is  a  major  problem  with  nonavailability  of  subject  matter  experts 
including  the  FPT  members.  We  don't  really  understand  [new]  systems  that 
well." 

"The  required  subject  matter  expertise  within  the  FPT  may  be  very  thin. 
Getting  people  who  are  really  experienced  with  advanced  systems  is  very 
di fficul t— you  can't  get  them  off  the  ships." 

"You  can't  expect  the  FPT  to  solve  any  problems  for  you— they  vary  too 
much  in  quality." 

"Fleet  Project  Team  continuity  is  a  problem.  Their  impact  varies  greatly 
with  particular  personalities." 

These  developer  and  user  comments  point  to  two  problems.  Naturally,  there 
is  a  problem  getting  top-notch  people  in  any  organization.  However,  there  is 
an  additional  problem  for  the  Fleet  Project  Team:  trainer  development  is 
often  to  support  a  weapons  system  acquisition  which  itself  is  in  an  early 
stage  and  may  incorporate  innovative  features  that  are  not  very  well  defined. 
Consequently,  it  is  difficult  for  the  FPT  members  to  act  effectively  in  the 
role  of  subject  matter  experts. 

"We  have  been  called  in  after  the  specifications  were  written  and  the 
contracting  accomplished  every  time." 
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"Quite  often  the  FPT  was  brought  in  in  an  acceptance  role  after  it's  too 
late.  The  [developer-laboratory]  loop  will  design  some  specifications 
without  any  FPT  input  whatever.  They  are  very  protective  of  their 
systems.  Laboratory  people  are  happily  isolated  from  the  fleet  users." 


Sometimes,  Fleet  Project  Team  members  like  the  ones  who  made  the  comments 
above  feel  their  views  aren't  all  that  welcome,  although  usually  they  seem 
interested  in  receiving  information  from  other  players.  We  believe  this 
represents  an  example  of  interactional  asymmetry.  As  mentioned  before,  the 
model  is  able  to  accomodate  and  represent  this  sort  of  relationship. 


"[The  Quality  Assurance  and  Revalidation  team]  tries  to  get  the  users, 
custodians,  and  operators  around  the  table — seems  to  be  the  only  time  they 
talk  to  each  other." 


This  comment  by  a  member  of  the  QA&R  team  suggests,  not  surprisingly,  that 
the  user  community,  like  the  other  organizational  elements  of  the  system,  has 
problems  with  internal  interaction.  Internal  interactions  are  represented  on 
the  diagonal  of  the  organizational  interaction  matrix,  Figure  5. 


FEATURES 


Up  to  this  point,  we  have  discussed  a  sampling  of  the  comments  made  during 
our  interviews  that  relate  primarily  to  organizational  factors,  which  comprise 
the  first  major  element  of  the  model  of  innovation  acceptance  described  in 
Section  Four.  Now,  we  wish  to  focus  attention  on  some  comments  which  were 
made  primarily  in  regard  to  training  device  features.  The  feature-specific 
aspect  of  acceptance  comprises  the  second  major  element  of  the  model. 

"Low-cost  cockpit  trainers  tend  to  fall  into  the  non-accepted  category. 
There  has  to  be  a  lot  of  fleet  understanding.  The  problem  is  that  even  if 
you  get  it  at  the  outset,  the  next  batch  of  users  may  not  be  accepting." 
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The  high  rate  of  personnel  turnover  in  the  Navy  poses  special  problems  for 
innovation  acceptance,  as  this  developer's  comment  suggests.  The  change 
advocate  (if  he  can  be  clearly  identified!)  is  likely  to  have  to  maintain  a 
continuing  involvement  after  innovation  introduction  to  facilitate  acceptance, 
rather  than  to  rely  on  "corporate  memory"  in  an  organization  with  such  a  high 
turnover  rate. 


"There  is  user  resistence  to  trainers  which,  because  of  their  complexity 
of  operation,  make  the  users  feel  like  a  dolt." 

"[The  developer's]  recommendations  on  the  specifications  are  critical  but 
they  tend  to  make  the  device  too  sophisticated.  We  don't  want  training 
devices  that  require  training  courses  to  learn  how  to  operate  them." 

"Users  don't  use  the  whole  device — often  don't  understand  its  full 
capabilities,  although  sometimes  what  they  don't  use  is  not  required  by 
their  curriculum." 

"One  elaborate  trainer  is  having  acceptance  problems  because  it  requires 
four  instructors.  What  role  did  the  FPT  play  in  permitting  this  to 
happen?" 

"We  buy  a  lot  of  trainers  with  features  that  are  never  understood  and 
never  get  used." 

"You  can  design  an  instructor  station  with  too  many  provisions  for 
feedback.  We  don't  need  buckets  of  statistics." 


These  developer  and  user  comments  address  the  issues  discussed  in  Section 
Four  under  the  heading  "Simplicity."  Often,  developers  and  builders  put  less 
emphasis  on  simplicity  than  users.  This  sometimes  leads  to  significant  user 
acceptance  problems. 

"The  end  user  is  the  guy  who  knows  the  least  about  what  trainer 

capabilities  are  required  to  meet  the  training  need." 

"Only  the  users  should  make  certain  kinds  of  tradeoffs— we  [the  users] 

should  decide  what  to  give  up  vs.  what  is  critical." 

This  pair  of  comments,  from  a  member  of  the  developer  community  and  a 

member  of  the  user  community,  respectively,  shows  an  interesting  disparity. 
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Organizational  interaction  can't  completely  resolve  this  sort  of  difference, 
but  without  a  reasonable  degree  of  interaction,  the  parties  can  become  very 
polarized,  indeed,  with  a  very  negative  impact  on  user  acceptance  of  the 
developer's  products. 

"There  are  [developer's]  engineers  who  are  strongly  opposed  to  trainer 
provisions  for  performance  evaluation." 

"The  users  don’t  know  how  to  use  performance  measurement  systems." 

"Instructors  may  be  resistent  to  trainer  features  such  as  performance 
measurement  systems  because  the  data  show  what  a  poor  job  they're  doing." 

"Performance  scoring  devices  may  not  be  accepted  as  well  as  'instant 
replay’  devices." 

"Fleet  personnel  are  resistent  to  automated  performance  evaluation 
systems. " 

These  funder,  developer  and  user  comments  highlight  an  area  where 
acceptance  problems  are  particularly  likely  to  be  experienced.  Automatic 
performance  measurement  is  sensitive  on  at  least  two  counts:  1)  the  validity 
of  the  methodology  employed  to  generate  specific  performance  measures  is  often 
open  to  question;  and  2)  the  performance  measures  are  often  threatening  to 
the  instructor  and/or  the  student.  Mutual  understanding  of  these 
considerations  among  the  concerned  parties  seems  to  us  a  necessary  condition 
to  paving  the  way  for  much  needed  innovation  in  performance  assessment. 

"Trainer  maintenance  is  a  major  factor  in  user  acceptance  problems." 

"There  is  inadequate  planning  for  life-cycle  support  of  trainers.  NTEC 
can  become  the  inheritor  of  devices  that  other  people  buy  with  no 
appropriate  ILS  [Integrated  Logistics  System],  technical  documentation, 
etc . " 

"Hardware  and  software  changes  to  trainers  come  extremely  late.  The  only 
thing  some  of  our  trainers  are  good  for  is  voice  procedures." 

"I  was  scheduled  for  100  part-task  training  hours  during  my  last  tour  and 
actually  got  15  [because  of  trainer  maintenance  problems]." 
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"Sometimes  the  WST  [Weapons  Systems  Trainer]  goes  down  after  you  are 
one-and-a-hal f  hours  into  a  problem.  There  is  no  way  to  recover  where  you 
were.  Thus  we  lose  qualification  time.  Not  many  of  our  crews  look 
forward  to  using  the  trainers  because  of  this." 

"OPNAV,  NAVAIR,  NTEC,  and  the  Fleet  work  closely  together  in  the  early 
stages  of  trainer  development  but  this  does  not  carry  over  into  the  life 
of  the  trainer." 

"Acceptance  problems  may  develop  over  time  because  of  corporate  memory 
loss  on  how  to  fully  operate  the  trainer." 

"Trainers  notoriously  degrade  with  operational  age." 

"If  a  trainer  is  not  maintainable  it  does  not  matter  how  good  the  design 
is  in  other  respects." 


As  these  funder,  developer,  and  user  comments  suggest,  user  acceptance  is 
not  something  which,  once  achieved,  is  assured  thenceforth.  It  is  a  condition 
which  requires  careful  attention  to  attain  initially,  but  it  also  requires 
continued  attention.  Truly,  achieving  user  acceptance  of  complex  technical 
systems  is  a  "cradle-to-grave"  proposition. 
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SECTION  SIX 


CONCLUSIONS  AND  RECOMMENDATIONS 


The  Navy  is  increasingly  dependent  upon  the  use  of  complex  training 
devices  for  the  achievement  and  maintenance  of  operational  readiness.  As  the 
scope  of  training  device  missions  has  increased,  so  has  the  cost  of 
procurement,  maintenance,  and  utilization,  placing  increased  importance  on 
effective  use.  However,  previous  research  has  shown  that  there  are  often 
important  mismatches  between  the  characteristics  desired  in  training  devices 
by  their  users  and  various  features  that  are  actually  designed  into  them. 

There  is  considerable  doubt  that  the  Navy's  research  effort  related  to 
training  innovations  is  achieving  its  full  potential  impact.  There  are 
substantial  differences  of  opinion  among  individuals  in  a  number  of  Navy 
organizations  relating  to  the  most  cost-effective  design  of  training  devices 
and  how  best  to  incorporate  innovative  developments  into  those  designs.  This 
results  in  problems  of  acceptance  on  the  part  of  both  individuals  and  Navy 
organizations,  whether  they  are  concerned  with  the  design,  development,  or  use 
of  training  devices.  Of  course,  acceptance  problems  at  the  point  of  the  user 
are  likely  to  be  the  most  consequential. 

In  an  effort  to  move  toward  a  solution  to  these  problems,  research  was 
conducted  to  develop  a  model  of  factors  influencing  user  acceptance.  The 
model  reflected  reviews  of  the  literature  on  innovation  acceptance,  technology 
transfer,  and  organizational  development;  review  of  official  directives  and 
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instructions  bearing  on  training  device  development,  acquisition,  and  use;  and 
interviews  with  personnel  representing  most  of  the  organizational  "players"  in 
the  trainer  development,  acquisition,  and  use  process. 


It  was  concluded  from  the  literature  on  innovation  acceptance  that: 


1.  There  is  a  wealth  of  information  concerning  variables  that  are  likely 
to  influence  the  acceptance  of  training  innovations,  the  most 
important  of  which  include  relative  advantage,  compatibility, 
simplicity,  observability,  trialability,  and  supportability.  An 
extended  discussion  of  this  information  is  presented  in  this  report. 

2.  No  predictive  model  has  been  developed  that  represents  both 

organizational  factors  and  feature-specific  factors  as  they  influence 
the  acceptance  of  innovations  of  any  kind. 


With  respect  to  the  literature  on  technology  transfer  in  the  Navy,  it  was 
concluded  that: 


1.  Many  important  factors  have  been  identified  and  incorporated  into  a 

useful  model  of  technology  transfer. 

2.  This  model  was  not  simple  enough  for  our  purpose  in  one  respect, 

namely,  that  some  of  the  factors  identified  were  not  represented  in  a 

form  that  would  facilitate  quantifying  or  scaling  them; 

3.  In  another  respect,  the  model  was  not  complex  enough,  in  that  it  did 
not  explicitly  recognize  that  a  variety  of  players  may  be  involved  in 
the  development,  acquisition,  and  use  of  innovations,  as  well  as  the 
interactions  and  feedback  loops  that  may  (or  should)  exist  among 
them; 

4.  However,  the  technology  transfer  model  was  useful  in  identifying  a 

number  of  important  processes  related  to  the  organizational  component 
of  the  model  we  developed. 


With  respect  to  the  organizational  development  literature,  it  was 
concluded  that: 


1.  Some  recent  efforts  to  relate  general  systems  theory  to 
organizational  development  were  pertinent  to  the  project  objectives, 
particularly  in  respect  to  the  emphasis  on  feedback  loops  within 
organi zations; 
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2. 


However,  no  model  was  found  that  provided  an  adequate  conceptual 
basis  for  identifying,  quantifying,  and  predicting  the  effects  of 
changes  in  feedback  loops  within  an  organisation,  such  as  the  Navy's, 
which  involves  development,  acquisition  and  user  suborganizations. 


It  was  concluded  from  a  review  of  official  Navy  directives  and 
instructions  pertinent  to  training  device  development,  acquisition,  and  use 
that: 

1.  It  is  difficult  to  obtain  a  coherent,  integrated  picture  of  the 
overall  training  device  acquisition  process  solely  from  the 
directives  and  instructions; 

2.  The  official  directives  and  instructions  represent  an  important 
adjunct  to  other  sources  of  information  in  that  they  prescribe  how 
things  are  supposed  to  be  done; 

3.  Not  all  of  the  directives  are  adhered  to  in  actual  practice. 


The  conduct  of  a  substantial  number  of  interviews  with  personnel  involved 
in  most  aspects  of  training  system  research,  development,  procurement, 
management,  and  use  led  to  the  conclusions  that: 


1.  Opinions  regarding  what  is  right  and  what  is  wrong  in  the  training 

device  acquisition  process  are  very  strong,  and  there  has  been  a 
recent  history  of  well  accepted  and  poorly  accepted  devices.  In  the 
words  of  one  respondent,  "almost  no  one  is  neutral  about  training 
devices." 

2.  The  interview  results  revealed  a  highly  complex  training  device 

acquisition  "organization";  in  fact,  organizational  makeup  varies 
widely  with  the  type  of  device  being  procured. 

3.  Very  different  points  of  view  were  expressed  by  members  of  the 

researcher,  funder,  developer,  and  user  communities.  Each  community 
appeared  governed  by  its  own  perspective,  while  holding  attitudes 
toward  the  other  communities  ranging  from  sympathetic  to 
antagonistic.  We  hypothesize  that  many  problems  of  innovation 
acceptance  are  caused  by  the  present  limited  interactions  and 
feedback  loops  among  these  communities. 


The  development  of  a  predictive  model  of  innovation  acceptance  led  to  the 
conclusions  that: 
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1.  The  model  should  have  two  major  components:  a  component  intended  to 
reflect  organizational  factors,  and  a  component  intended  to  reflect 
specific  features  of  innovations; 

2.  The  model  should  permit  explicit  representation  of  interactions  among 
all  of  the  numerous  organizations  that  are  involved.  We  hypothesize 
that  the  categories  of  researcher,  funder,  developer,  builder,  and 
user  are  sufficient; 

3.  The  model  should  permit  representation  of  interactions  and  feedback 

loops  between  organizational  elements  on  a  number  of  relatively 

simple  factors  designed  to  permit  quantification,  by  the  use  of 
survey  techniques  and  psychometric  methods.  Nine  organizational 
interaction  factors  were  presented,  which  were  hypothesized  to  be 
sufficient; 

4.  Six  features  of  innovative  devices  were  identified  which  are 

hypothesized  to  be  particularly  important  in  the  innovation 
acceptance  process,  and  which  were  included  in  the  feature-specific 
component  of  the  model ; 

5.  The  model  should  be  comprised  of  a  weighted  linear  combination  of  the 
factors,  at  least  in  this  first  approximation,  since  available  data 
do  not  support  proposing  a  higher-order  (non-linear)  model. 

6.  The  model  has  been  found  to  be  useful  in  structuring  thinking  about 

problems  of  innovation  acceptance.  It  is  hypothesized  that  it  will 

be  useful  in  identifying  organizational  processes  that  might  benefit 

from  change,  and  in  identifying  areas  where  future  research  on  the 
acceptance  process  may  have  the  greatest  impact; 

6.  Efforts  should  be  made  to  quantify  at  least  some  aspects  of  the 

model,  to  assess  its  utility  and  to  attempt  to  produce  results  which 
may  be  applied  directly  to  solve  or  reduce  some  of  the  practical 
problems  of  innovation  acceptance  identified  during  this  study. 
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APPENDIX  A 


OFFICIAL  DIRECTIVES  AND  INSTRUCTIONS 


There  are  many  official  directives  and  instructions  bearing  on  the 
training  system  acquisition  process.  The  titles  of  those  we  have  reviewed  are 
listed  below.  They  are  listed  by  issuing  agency. 


CNET 


ltr  N-E,  Test  and  evaluation  procedures  for  training  devices,  6  Nov  1979. 

Guidelines  N-945,  Submarine  Trainer  Working  Group,  26  Nov  1980. 

ltr  N-945,  Draft  SECNAV/OPNAV  Instructions  on  Selection  of  Training 
Equipment;  comments  concerning,  15  Apr  1981. 

INST  1500.9,  Participation  by  the  Naval  Education  and  Training  Command  in 
the  preparation  and  implementation  of  Navy  Training  Plans,  26  June  1974 

INST  1551.5A,  Training  Situation  Analyses  (TSA);  procedures  for 

conducting,  3  Jan  1978. 

INST  3920. IB,  Research,  Development,  Test  and  Evaluation  Program  of  the 
Naval  Education  and  Training  Command,  26  Aug  1981. 

INST  5450.8  ( NAVMAT  INST  5450.28),  Additional  duty  functions  of  the 
Commanding  Officer,  NAVTRAEQUIPCEN,  Orlando,  Florida,  to  CHNAVMAT  and 
relationships  between  NAVTRAEQUIPCEN  and  Systems  Commanders,  Project 
Managers  and  others,  14  Dec  1972. 

INST  7000. 2A,  Procedures  and  Responsibilities  for  the  development  of  the 
Program  Objective  Memorandum  (POM)  documentation  for  the  CNET  Training 
Device  program,  22  June  1979. 

INST  7000. ?B,  Policy,  procedures,  and  responsibilities  for  the  planning 
and  resource  requirement  identification  for  surface  and  subsurface 
training  devices,  (draft). 

MEMO  N-5,  Test  and  Evaluation  of  Training  Devices,  10  Feb  1978. 

NAVCOMPT 


Memo  NCB  521,  Requirement  for  budgeting  procurement  of  first  item  training 
devices  in  RDT&N,  14  Dec  1977. 
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NAVMAT 


INST  1500.12,  Technical  Training  Equipment  Acquisition  Requirements,  6  Apr 
1981. 

INST  4720.1,  Approval  of  systems  and  equipments  for  service  use  (ASU),  13 
Dec  1974. 

INST  5450.28  (CNET  INST  5450.8),  Additional  duty  functions  of  the 
Commanding  Officer,  NAVTRAEQUIPCEN,  Orlando,  Florida,  to  CHNAVMAT  and 
relationships  between  NAVTRAEQUIPCEN  and  Systems  Commanders,  Project 
Managers  and  others,  14  Dec  1972. 

NAVSEA 

INSTR  1543.1  A,  Manpower,  Personnel,  and  Training  Support  for 
NAVSEA — Cognizant  Ship,  System,  Equipment,  and  Non-hardware  Developments. 

NAVTRAEQUIPCEN 

Itr  N-8/121,  Test  and  Evaluation  procedures  for  training  devices  (Draft 
CNET  INSTRUCTION  3960.  "Test  and  Evaluation  Procedures  for  RDT&E  Training 
Device  Development"),  28  Feb  1980. 

INST  1551.7B,  Fleet  participation  in  development  acquisition,  and 
acceptance  of  major  training  devices,  18  Jun  1974. 

INST  3900. IOC,  The  Naval  Training  Equipment  Center  planning  and 
programming  system;  establishment  of,  29  Aug  1977. 

INST  3910. 4A,  Functional  Statement,  Functional  Description,  Mini -Mi li tary 
Characteristics,  and  Detail  Military  Characteristics;  instructions  and 
responsibilities  for,  18  Feb  1977. 

OPNAV 

Memo  901/582126,  Programming  Guidance  for  the  Acquisition  of  Training 
Devices,  14  March  1978. 

Itr  992F2/640753,  Conference  on  Early  Identification  of  New  Systems, 
Equipment,  and  Other  Developments  Generating  Training  Requirements; 
reports  of,  29  Mar  1976. 

INST  1500.8H,  Preparation  and  implementation  of  Navy  Training  Plans  (NTPs) 
in  support  of  hardware  and  non-hardware  oriented  developments,  3  Jul 
1  975. 

INST  1551. 7B,  Fleet  participation  in  development,  acquisition,  and 
acceptance  of  major  training  devices,  22  Apr  1977. 

INST  10171.5,  Certification  of  Major  Aviation  Training  Devices,  5  Sep  1978. 
MEMO  987/140521,  Independent  Evaluation  of  Training  Devices,  30  Dec  1977. 


100 


MEMO  991B/644005,  "Operational"  Test  and  Evaluation  of  Training  Devices, 
10  Jan  1978. 

MEMO  3SC/18,  Evaluation  of  Traning  Devices,  13  Jan  1978. 

Guide,  Management  of  Manpower,  Personnel  and  Training  Research, 
Development,  Studies,  and  Surveys,  (draft)  3  June  1981. 

ltr  29P/384170,  Submarine  Training/Trainer  Working  Group,  16  June  1982. 

SECRETARY  OF  THE  NAVY 


INST  9089.1,  Selection  of  Trainers/Training  Devices,  9  June  1980. 
OTHER 


Audit  Report  1400058,  Servicewide  Audit  of  the  Aircraft  Flight  Simulator 
Program,  4  Jan  1980. 
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APPENDIX  B 


TRIPS  AND  INTERVIEWS  CONDUCTED  DURING  THE  PROJECT 

An  attempt  was  made  to  conduct  personal  interviews  with  representatives  of 
all  Navy  organizations  involved  in  the  training  device  acquisition  and  use 
processes.  All  major  platforms  (air,  surface,  submarine)  were  represented  and 
a  wide  variety  of  training  devices  were  used  as  a  focus  for  discussion.  The 
locations,  offices,  and  training  devices  included  in  the  survey  are  listed 
below.  At  least  one  Fleet  Project  Team  member  was  interviewed  for  each  of  the 
training  devices  listed. 


ORLANDO 

o  Air  Planner,  NTEC  Plans  and  Programs  Group 
o  NTEC  Aviation/EW  Analysis  and  Design 
o  Training  Analysis  and  Evaluation  Group 
o  Sea  Planner,  NTEC  Plans  and  Programs  Group 
o  Director,  Research  Department,  NTEC 
o  Human  Factors  Laboratory,  NTEC 
o  Advanced  Simulation  Concepts  Laboratory,  NTEC 

PENSACOLA 

o  Assistant  Chief  of  Staff,  Training  Systems  Management,  CNET 
o  Surface  Combat  Systems  Devices,  CNET 
o  Quality  Assurance  and  Revalidation,  CNET 
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WASHINGTON,  D.C. 

o  Director,  Weapons  Training  Division,  NAVAIRSYSCOM 
o  Psychologist,  Weapons  Training  Division,  NAVAIRSYSCOM 
o  Office  of  the  Deputy  Undersecretary  of  the  Navy  (Research  and  Advanced 
Technology) 

o  Office  of  the  Deputy  Chief  of  NavalOperations,  Aviation  Manpower  and 
Training  Division 

o  Naval  Sea  Systems  Command,  Submarine  Combat  Systems  Project  Office 
o  Office  of  the  Deputy  Chief  of  Naval  Operations,  Submarine  Manpower  and 
Training  Requirements  Division 


0 

Office  of 

the  Deputy  Chief  of  Naval  Operations  (Manpower,  Personnel 

Traini ng) 

RVAW-110,  NAS 

MIRAMAR 

0 

15F8A 

E-2C 

Tactics  Part  Task  Trainer 

VF- 

-124,  NAS  MIRAMAR 

0 

2F95 

F-14A 

Operational  Flight  Trainer 

0 

15C9A 

F-14A 

Mission  Trainer 

0 

2F112 

F-14A 

Weapons  Systems  Trainer 

FLEET  ASW  TRAINING  CENTER,  PACIFIC 

0 

16H3 

Navy  Tactical  Game-Based  Training  System 

0 

14A12 

Surface  ASW  Trainer 

0 

2CA66 

ASW  Tactical  Team  Trainer 

0 

14E37 

SQR-17 

Team  Training  Simulator 

0 

14E38 

SQR-18 

Team  Training  Simulator 

0 

14E35 

SQQ-89 

Acoustic  Operator  Trainer 

0 

14E31B 

SQS-56 

Basic  Sonar  Operator  Trainer 

FLEET  COMBAT  TRAINING 

CENTER,  PACIFIC 

0 

20F16 

Tactical  Action  Officer  Trainer 

0 

7B4 

OUTBOARD  Trainer 

COMMANDER,  ASW  WINGS, 

PACIFIC 

0 

14D1 

ASW  Basic  Operator  Trainer 

0 

14B49 

S-3A 

Acoustic  Sensor  Operator  Trainer 

0 

2F92A 

S-3A 

Weapon  Systems  Trainer 

0 

14E10 

SH-3 

Sonar  Operator  Trainer 

0 

2F64C 

SH-3 

Weapons  Systems  Trainer 

0 

DASTS 

Acoustic  Training  Stimulator 
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HSL-31  and  FASO,  NAS  NORTH  ISLAND 
o  2F106  SH-2F  Weapons  Systems  Trainer 

o  2F135  SH-60B  Operational  Flight  Trainer 

o  2F117B  CH-46E  Operational  Flight  Trainer 


VS— 41 ,  NAS  NORTH  ISLAND 
o  14B49  S-3A 

o  14B50  S-3A 

o  2FS2A  S-3A 


Acoustic  Sensor  Operator  Trainer 
Part  Task  Trainer 
Weapon  Systems  Trainer 


In  addition  to  the  Fleet  Project  Team  members  for  the  above  devices,  we 
interviewed  the  manager  and  field  inspectors  of  CNET's  Quality  Assurance  and 
Revalidation  West-coast  office  regarding  their  program  of  training  device 
inspection  and  their  interfaces  with  other  players  involved  in  trainer 
development,  use,  maintenance,  and  continuing  support.  We  also  interviewed 
the  enlisted  training  officer.  Fleet  ASW  Training  Center  Pacific;  ASW  Training 
Officer,  SURFPAC;  Commanding  Officer,  SUBTRAFAC;  Assistant  Chief  of  Staff  for 
Training,  SUBGRU5;  and  3  persons  from  ASWINGSPAC  and  ASW  Squadron  VS-33. 


Finally,  we  interviewed  Dr.  Chuck  Jorgenson,  ARI  Field  Unit,  Fort  Bliss, 
who  gave  generously  of  his  time  so  that  we  might  better  understand  the 
training  system  acquisition  process  employed  in  the  Army.  Dr.  Dan  Fulgham  of 
Technology,  Inc.,  San  Antonio,  Texas  graciously  provided  us  with  an  overview 
of  U.S.  Air  Force  training  acquisition  processes. 
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APPENDIX  C 


EVENTS  AND  DOCUMENTS  RELATING  TO  FIRST  ARTICLE  TRAINING  DEVICE  DEVELOPMENT 

There  is  a  great  deal  of  variability  in  training  device  development 
depending  on  the  specific  trainer  involved.  Events  and  documentation  vary 
greatly  depending  on  trainer  sponsor,  trainer  development  agency,  and  on  the 
specific  issues  of  each  unique  development.  Furthermore,  directives  and 
instructions  which  impact  on  the  first  article  training  device  development 
process  are  in  a  state  of  flux,  so  that  the  procedures  followed  in  the 
documentation  required  "in  principle"  are  constantly  changing.  For  this 
reason,  it  is  difficult  to  provide  a  generalized  overview  of  how  the 
development  process  works  now. 

An  approach  which  may  prove  useful  in  understanding  the  components  of  the 
process  is  to  describe  a  systematic  sequence  of  events  and  documents  for  first 
article  training  device  development  which  has  been  recommended  for 
implementation  (Nutter  and  Terrell,  1S82).  This  sequence  was  developed  by  the 
Training  Analysis  and  Evaluation  Group  at  the  request  of  the  Chief  of  Naval 
Education  and  Training.  We  cannot  judge  whether  this  is  the  optimal 
management  scheme  for  training  device  development,  but  it  does  represent  a 
well  thought  out  approach  that  appears  to  be  sound  and  systematic. 
Furthermore,  most  activities  which  currently  are  engaged  in  a  trainer 
development  can  be  understood  within  the  context  of  the  TAEG  management 
system,  although  it  must  be  understood  that  specific 
developments  go  through  development  steps  which  are  analogous  to  a  selected 
subset  of  the  TAEG  model . 
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The  TAEG  management  model  is  represented  schematically  in  Figure  A-l , 
which  is  from  Nutter  and  Terrell,  1982.  We  will  provide  a  brief  description 
of  each  of  the  elements  of  this  model  which  is  paraphrased  from  Nutter  and 
Terrell's  more  detailed  descriptions.  Although  we  have  made  every  effort  not 
to  distort  those  authors'  intentions  in  providing  this  abridged  discussion,  we 
recommend  highly  that  the  reader  study  the  original  source  to  ensure  complete 
understand!'  ng. 

Each  event  and  each  document  is  described  in  subsequent  paragraphs. 
The  titles  of  the  events  and  documents  correspond  to  those  used  in  the  Figure 
where  appropriate  specific  approval  or  review  functions  are  indicated;  most 
documents  are  described  in  general  functional  terms  and  their  specific  format 
and  content  is  still  in  development. 

Event  1  Activity.  Training  Device  Requirements  are  formally  documented 
by  the  Navy  Training  Plan  and  the  Development  Plans  of  major  weapons  systems. 
In  addition,  any  fleet  activity  or  Navy  command  may  identify  a  training 
requirement  and  submit  it  to  the  appropriate  sponsor  in  the  form  of  a 
documented  Training  Requirement  Statement  (TRS)  via  the  chain  of  command. 

Document  1.1  Training  Requirement  Statement  TRS).  The  TRS  is  a 
proposed  document  modeled  after  the  Mission  Element  Need  Statement  (MENS). 
Training  Requirement  Statements  are  concise  statements  of  current  or 
anticipated  training  deficiencies  or  needs  that  have  a  high  perceived 
probability  of  solution  with  the  use  of  some  type  of  training  aid  or  device. 
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Event  2  Resource  Sponsor.  OPNAV  resource  sponsors  are  the  terminal 


points  of  TRS's  submitted  by  fleet  activities  or  Naval  commands.  The  sponsors 
establish  priorities  for  TRS's. 

Event  3  SWTG  and  STWG.  The  SWTG  or  STWG  determine  the  initial 
acceptability  of  the  TRS  for  further  development.  Since  it  does  not  have  an 
advisory  group  corresponding  to  the  SWTG  and  STWG,  0P01  performs  the  actions 
included  in  event  3  for  0P01  projects  [and  presumably,  0P05  performs  them  for 
OPOE  projects.] 

Event  4  Chief  of  Naval  Education  and  Training  (CNET).  CNET  tasks  the 
analysis  agent  (AA),  normally  NTEC,  to  perform  a  preliminary  front  end 
analysis  related  to  the  needs  addressed  in  the  TRS.  The  AA  working  with  the 
advice  and  input  of  concerned  activities  (event  4b),  provides  for  the 
performance  of  the  analysis  and  submits  to  CNET  a  Training  Requirement  Needs 
Analysis  (TRNA)  for  review  and  approval. 

Document  4.1  Training  Requirement  Needs  Analysis  (TRNA).  The  TRNA  is  a 
proposed  document  modelled  after  the  intent  of  the  Milestone  0  decision 
point.  The  TRNA  serves  as  the  baseline  document  for  development  of  subsequent 
program  documents.  The  primary  objective  of  the  TRNA  is  to  provide  a 
preliminary  assessment  of  the  training  situation  including:  a  statement  of 
the  training  problem,  training  objectives,  tentative  solution  (which  may 
include  a  training  device),  and  budget  estimates. 

Event  5  Resource  Sponsor.  The  TRNA  is  submitted  by  CNET  to  the  OPNAV 
resource  sponsor.  Resource  sponsor  approval  of  the  TRNA  signifies 
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establishing  the  training  requirement  and  commits  resources  to  formally 
develop  a  Training  Operational  Requirement  (TOR).  Resource  sponsor  criteria 
for  approval  include:  validity  of  training  requirement,  perceived  need  for  a 
solution,  priority  of  the  training  requirement,  and  funding  availability. 

Event  6  Chief  of  Naval  Education  and  Training  (CNET).  The  resource 
sponsor  tasks  CNET  to  prepare  a  TOR.  Normally,  the  TOR  would  be  prepared  by 
the  AA  (Event  6a)  and  forwarded  to  CNET  for  concurrence.  The  TOR  is  submitted 
in  draft  form  to  the  Resource  Sponsor  for  official  issue. 

Document  6.1  Training  Operational  Requirement  (TOR).  The  TOR  is  the 
proposed  training  device  version  of  the  operational  requirement  (OR).  The  TOR 
not  to  exceed  three  pages,  is  a  concise  statement  of  training  needs.  It  is 
the  basic  requirement  document  for  all  Navy  training  acquisition  programs 
requiring  research  and  development  effort.  The  TOR  directs  the  Chief  of  Naval 
Material  to  prepare  a  Training  Decision  Alternative  Proposal  (TDAP). 

Event  7  Resource  Sponsor.  The  OPNAV  resource  sponsor  approves  the  TOR 
(Document  7.1)  and  issues  it  with  a  request  for  development  of  a  Training 
Decision  Alternative  Proposal  (TDAP). 

Event  8  Chief  of  Naval  Material  (NAVMAT).  The  Chief  of  Naval  Material 
may  designate  NTEC,  under  the  command  of  CNET,  to  serve  as  analysis  agent  to 
plan  and  execute  projects  for  developing  training  material.  The  AA  prepares 
the  TDAP  for  CNET  concurrence  and  submission  to  the  resource  sponsor  via  the 
CHNAVMAT. 
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Document  8.1  Training  Decision  Alternatives  Proposal  (TDAP).  The  TDAP 


is  the  proposed  training  community  version  of  the  Development  Proposal  (DP). 
The  TDAP  subsumes  the  TR&A  and  contains  updated  information  and  more  indepth 
analysis  related  to  those  elements  which  describe  the  alternate  solutions  and 
trade-offs  designed  to  achieve  a  particular  range  of  capabilities  in  response 
to  the  TOR.  Cost-benefit  data  and  technical  considerations  are  provided  to 
assist  the  resource  sponsor  in  selecting  from  among  the  specified  alternatives. 

Event  9  Resource  Sponsor.  The  OPNAV  resource  sponsor  selects  an 
alternative  from  among  those  described  in  the  TDAP  or  returns  the  TDAP  for 

additional  analysis.  If  the  TDAP  is  approved,  the  resource  sponsor  tasks  the 
development  agent  to  develop  the  selected  alternative  and  submit  a  Training 

Equipment  Test  and  Evaluation  Plan  (TETEP)  (Document  10.1)  for  approval.  The 
TETEP  is  analogous  to  the  the  previously  employed  NDCP  (Navy  Decision 

Coordinating  Paper)  and  TEMP  (Test  and  Evaluation  Master  Plan). 

Event  10  Development  Agent  (DA).  The  DA  usually  Naval  Training 

Equipment  Center  (NTEC),  prepares  the  TETEP.  The  DA  recommends  to  the 
Resource  Sponsor,  at  the  earliest  possible  date,  the  formation  of  a  Fleet 
Project  Team  (FPT). 

Document  10.1  Training  Equipment  Test  and  Evaluation  Plan  (TETEP). 
The  TETEP  is  the  proposed  training  community  version  of  the  the  TEMP.  The 
TETEP  includes  the  information  formally  covered  in  NDCP  as  well  as  test  and 
evaluation  plans,  and  replaces  the  NDPC  as  the  Program  Initiation  Document  for 
the  acquisition  process.  The  TETEP  and  its  two  required  appendices,  the 
Training  Equipment  Functional  Baseline  (TEFB)  and  the  Training  Facilities 
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Requirement  (TFR),  basically  form  a  contractual  agreement  among  the 

development  agent,  the  testing  agents,  the  Naval  Facilities  Engineering 
Command,  and  the  CNET.  This  relationship  is  approved  by  the  resource  sponsor. 

Event  11  Chief  of  Naval  Education  and  Training  (CNET).  The  CNET,  as 

training  agent,  specifies  and  approves  the  provisions  in  the  TETEP  which 
relate  to  training.  The  CNET  recommends  approval  and  forwards  TETEP  to  the 

resource  sponsor. 

Event  12  Resource  Sponsor.  When  the  Resource  Sponsor  approves  the 

TETEP,  it  officially  authorizes  the  program  start.  Approved  TETEP's  are  sent 
to  CNO  (OP-098,  Director,  Research,  Development,  Test  and  Evaluation)  for 
promul gation . 

Event  13  CNC  (OP-098).  OP-098  acts  for  OPNAV  as  sponsor  for  the  RDT&E 

appropriation  and  manages  the  planning  and  reporting  procedures  followed 
during  the  conduct  of  RDT&E.  OP-098  also  coordinates  the  formulation  and 

promulgation  of  RDT&E  requirements,  appraises  the  progess  of  RDT&E  effort,  and 
where  appropriate,  recommends  projects  for  curtailment,  suspension,  or 
cancellation.  Finally,  OP-098,  as  the  OPNAV  focal  point  for  TETEP,  reviews 
them  for  adequacy  of  planned  operational  testing,  including  funding  and 

scheduling.  Promulgation  of  the  TETEP  and  updated  versions  by  OP-098 

indicates  acceptance  of  the  RDT&E  process  to  that  point  in  the  program  and 
authorizes  the  continuation  of  development  efforts. 

Event  14  Development  Agent  (DA).  The  DA  prepares  requests  for 

proposals  (RFP)  based  on  the  TEFB  (Training  Equipment  Functional  Baseline) 
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[analagous  to  the  military  characteristics  MC  document]  and  supporting 
documents,  negotiates  the  contract(s),  monitors  and  coordinates  contractor 
development  progress,  and  conducts  development  tests  as  specified  in  the 
TETEP. 


Event  15  Contract  Development.  This  event  represents  that  period  of 
time  required  for  preparation  of  contractual  documents  and  the  time  required 
for  the  training  device  development  and  fabrication.  [During  this  time  the 
development  agent  and  the  fleet  project  team  have  a  very  important  role  in 
assuring  that  a  suitable  product  is  being  developed.] 

Event  16.  Development  Tests  (DT).  Development  tests  are  conducted 
throughout  the  different  stages  of  actual  development  of  the  prototype  device 
by  the  DA  in  concert  with  the  fleet  project  team.  Development  tests 
demonstrate  that  the  engineering  design  meets  performance,  maintainability, 
supportabil ity,  environmental  compatabil ity ,  and  system  safety  requirements  as 
stated  in  the  TEFB  (Training  Equipment  Functional  Baseline).  Development 
tests  may  include  contractor  tests  and  Navy  acceptance  tests.  The  final  DT 
phase  is  usually  on-site  acceptance  testing,  the  purpose  of  which  is  to 
certify  that  the  device  meets  all  technical  requirements  specified,  and  is 
ready  for  "Ready  for  Training"  testing. 

Event  17.  Ready  for  Training  (RFT).  Ready  For  Training  certifies  that 
the  training  device,  installation,  logistic  support,  training  syllabus,  lesson 
plans  and  instructor  training  requirements  as  defined  in  the  contract  have 
been  met  and  the  device  is  ready  for  training. 


112 


Event  18  Training  Capability  Test  (TCT).  Following  certifcation  of  RFT 
the  device  is  submitted  to  a  TCT  by  an  independent  agent.  Independent  agent 
is  defined  as  a  command  or  agency  independent  of  the  developing,  procuring, 
and  using  commands. 

Event  1?  Approval  for  Training  Acceptance  (ATA).  Devices  which  are 
certified  RFT  and  which  meet  the  requirements  defined  in  the  TEFB  (Training 
Equipment  Functional  Baseline)  are  certified  ATA.  Once  a  device  is  certified 
ATA,  follow-on  devices  may  be  procurred  throughout  the  training  command. 
[Obviously,  there  results  a  great  pressure  to  obtain  prompt  certification  of 
ATA.] 


Event  20  Training  Effectiveness  Evaluation  (TEE).  CNET  may  require  a 
TEE.  This  decision  must  be  made  early  in  the  acquisition  cycle  so  that  the  DA 
may  include  planning  and  budgeting  for  the  TEE  in  the  TETEP.  The  TEE  is 
performed  by  an  independent  agent  who  determines  whether  the  device  is 
effective  in  training  students  to  satisfy  their  course  objectives  and  in  rare 
cases  whether  the  training  is  transferrable  to  performance  on  the  job. 
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